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Summary 



This study emerges from a conviction that new media have much of value to 
offer to education, but that there are problems in deciding where and how 
to use them, in what combinations, under what conditions, and for what 
purposes. A review of the multi-media field (including standard audio and 
visual devices, television applications, computer-assisted instruction, 
instructional data-transfer systems, and so on) indicates that current 
practices are generally rather ineffective and that research and develop- 
ment efforts have be in limited in scope. New approaches are evidently 
required. 

/The feasibility and desirability of laboratory approaches to educational 
(media were investigated by this project^ ^Systems methods formed the 
background of the investigation, though the project was limited to 
conceptual exploration^ and to the analysis of past and present experience. 
No new laboratory was constructed for trial and evaluation, but (expert 
opinions representing many disciplines and backgrounds were sought out, 
written accounts were reviewed, and innovative installations were 
visited^ 

An important conclusion of the study is that "multi-media problems" 
neither exist nor can be solved in isolation from total educational 
systems, including their political and social environments. A labora- 
tory to address these problems should be concerned with a complete 
spectrum of activities, including research, system planning, design, 
operation, evaluation, and diffusion. 

Six types of laboratories are distinguished as being able to deal in 
various ways with educational media. Of these, simulation laboratories 
and ("comprehensive development" laboratories^fehow greatest promise of 
offering needed assistance. 

Simulation laboratory approaches to educational problems have to date 
"“shown limited success in relation to costs. A semi -manual, computer- 
based laboratory utilizing time-sharing techniques is judged to have 
the greatest chance of improving simulation laboratory effectiveness, 
but its main contribution would be as a research vehicle. It would tend 
to bring appropriate models, data, and functioi al analyses into existence 
as both a condition and a result of its operation, but it could do little 
by itself toward reaching solutions to the major problems which lie in 
the areas of development and diffusion. It therefore does not represent 
a feasible approach to multi-media problems. 

The concept of a "comprehensive multi-media development laboratory'^is 
put forward. It would plan, supervise, test, and support systematic! 
educational improvements in an integrated and continuous fashion. It 
would review, analyze, evaluate, and generate alternatives to current 
multi-media practices and to the assumptions and institutions in which 
they are embedded. It would be ^organized around methodology and functions 
rather than around equipment and facilities; techniques would be sub- 
ordinated to educational problems., 

V 
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A comprehensive multi-media development laboratory as described is needed, 
is technically feasible, and would have an excellent probability of mak- 
ing significant contribution?, to education without excessive costs. It 
is recommended that such a ]abo7:atory be established. 

In view of the recent formation of a Commission on Instructional Techno- 
logy under Title III of the Public Broadcasting Act of 1967, it is noted 
that the comprehensive development laboratory concept has considerable 
relevance to needs for improving the educational effectiveness of instruc- 
tional television and radio. If a new overall approach to telemedia is 
desired, an organization along the lines of the comprehensive development 
laboratory could combine, a® no other organization could, all the factors 
necessary for a complete approach. 
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I. Introduction 



A. Project Purpose and Background 

This document is the final report of a study performed by System Develop- 
ment Corporation (SDC), under contract to the Bureau of Research of the 
U. S. Office of Education, to determine the feasibility of using an 
experimental laboratory for identifying classroom multi-media problems 
and requirements. The study emerges from a conviction that new media 
have much of value to offer to education, but that there are problems in 
deciding where and how to use them, in what combinations, under what 
conditions, and for what purposes. The study has attempted to discover 
what type of laboratory approach could best assist in finding solutions to 
these problems. 

Multi-media techniques for education include telemedia ted resources, 
computer applications, the myriad devices for classroom project, n of 
sight and sound, individual teaching devices, and some advances not yet 
implemented. They appear worthy of serious consideration, both because 
of their great promise and because important uncertainties surround their 
current and future use. More specifically, multi-media appear to offer 
the following kinds of advantages if properly implemented: 

- Improved use of the nation’s limited educational resources. 

- Increase in student-teacher ratios without a lessening in the quality 
of education. 

- Relief from the tedium which marks the classroom pattern of tradi- 
tional education. 

- Increased student involvement in the design and control of the learning 
situation, and increased active participation in and responsibility for 
learning. 

- Improvement in the dissemination of quality education, the updating 
of needed skills, and the provision of educational opportunities for 
populations of increasing size, variety, and dispersion. 

However, it also appears that multi-media will significantly improve 
education only to the degree that their total impact is perceived and 
planned for. They can change the whole concept of the institution of 
education and the schools through their revolutionary power, range and 
potential. On the other hand, they may have no effect on the schools 
just because massive institutional inertia can too easily diffuse the 
'uncoordinated impact of random ideas and sporadic innovative trials. 

Computers, for example, have received considerable notice in the popular 
press during the past several years; in particular, computer-assisted 
instruction has been hailed as a method of providing individualized 
instruction. Indeed, computer applications to education have been 
viewed as making it possible to convert education from a labor-intensive 
to a capital-intensive industry. As greater numbers of students enter 
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public education, it has been predicted that correspondingly increased 
numbers of trained teachers will not be necessary; it will only be 
necessary to attach additional student terminal equipment to the 

computer. 

Although computer technology shows some promise of increasing the 
efficiency of education by individualizing instruction and converting i 
to a capital-intensive system, thus far it has not delivered on its 
promise. In addition, the prospect of computer-assisted instruction 
raises serious problems on a larger scale. First, efficiency is not 
always a major goal of public education in an affluent society; it may 
even create problems. Second, individualized instruction may be in- 
consistent with the equalitarian movement in the United States and with 
education’s implicit "custodial" or "babysitting functions. Third, 
teachers represent a powerful lobby group that does not look with favor 
on any effort to replace them with technology. 

Nevertheless, despite these obstacles, technology offers the capability 
of achieving a better allocation of educational resources. Clearly tovnnan 
instructors are poorly used as simple transmitters of information^ Radio 
broadcasts, continental distribution of educational telecasts by means of 
communication satellites, widescale distribution of copying machines, 
audio tape equipment, and the dissemination of Instructional material on 
microform copy with associated readers, are but a few of the methods that 
could handle the bulk of the information-passing function that is now 
the nearly unchallenged domain of the instructor and his blackboard. 

A better use of the instructor could also be made by further differen- 
tiating his rol_£* The instructor could be a manager of instructional 
resources rather than a transmitter of subject matter. \He could direct 
the learner to appropriate learning resources (programmed instructional 
material, educational games, and so on) and become a partner with the 
student in the learning process., Greater use can also be made of 
another major instructional resource: the students themselves, students 

often learn more from each other than from teachers and can be taught to 
help one another. Older children who help younger children also benefit 
themselves. This opportunity has been neglected in teacher-training 
efforts, yet the students themselves represent the single most important 
hidden resource available for the reduction of the teacher-pupil ratio. 

It is also unnecessary to assume that every instructor must be a fully 
trained professional. Many .parents or para-professional aids may be 
used for such functions as preparing and maintaining instructional materi , 
working in the home showing the parent how to help the child, 1 keeping 
records, tutoring, monitoring student activities, scoring tests, or taking 

students on field trips. 

The uses of media and of differentiated staff roles as described above 
are methods of dealing with (the economic problem of processing increasi g 
numbers of students in the face of inadequate numbers of trained instruc- 
tors. But these measures say little about improving the aualityof 
learning. Yet nothing in the research literature reviewed for this projec. 
(see Appendix A) nor any experience of the project’s staff and consultants, 
contradicts the conclusion that the only known method of improving the 



Quality of learning consists of defining a set of learning objectives, 
evaluating discrepancies between those objectives and the performanc 
of the instructional products and procedures, and continually changing 
the instruction to minimize the discrepancy. This evaluation-revision 
concept assumes that no instructional system is perfect, but that 
progressive improvement can be expected from repeated use of evaluation 
and revision cycles. (The evaluation-revision cycle is a built-in self 
correcting mechanism for quality control of the educational system^ 

The foregoing paragraphs are brief reminders that today educational 
theory and practice are both in a state of turmoil. Media have been a 
UTof much of the discussion and of many innovative trials. For this 
reason, it has appeared of some urgency to devise-if possible-a 
laboratory approach toward making effective use of multi-media. 

B. Some Working Definitions 

An important assumption of this project has been that the key terms 
"multi-media" and "experimental laboratory" were not to be understood 
narrowly. For this reason, and since the study has strongly emphasized 
systems aspects (so that not just media devices b f n f ‘‘If 

tional contexts have been considered) , precise and limiting definitions 

have not been framed. 

For the purposes of this report, the term multl-me j i a includes primarily 
the newer devices for presenting audio and visual stimuli and for 
collecting and recording responses, especially in combined, complex, or 
expensive forms. Examples include television (whether broadcast closed- 
circuit, or videotape) .'student response indicators, computer-assisted 
instruction, language laboratories, and audio-visual-textbook course 
packages. "Telemedia" include television, radio, telelecture, and 
instructional data-transfer applications. Some of these “c no 
have become relatively commonplace, but some exist primarily a *P erl 
mental versions. The emphasis in this study has been on the newer 
instructional technologies— those which are making possible and are 
requiring (or will require) radical changes in the basic structure of 
the learning situation. For these devices, special methods such as 
are studied by this project are necessary. 

The meaning of [experimental laboratory has emerged during the study 
rathe^than bei^set down as pre-edition. The If oratory which s 
ultimately described and recommended as feasible and highly desirabl • 

is quite unlike what laboratories are conventionally assumed to be. 
contains no workshops, equipment, computers, experimental subjects, 
or white-coated assistants. Instead, it is defined as having /a. program 
^methodology , and resources for application to making major improvements 

in education^ 

It is clear that a number of laboratories for the stud y °f Particular 
aspects of multi-media problems already exist. In fact , vir tual ly y 
university and each producer of components or software maintain some 
form of laboratory for research ia ‘heir particular area of interest. 

But these laboratories have made little headway in defining effective 
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roles for media. A reorientation of thinking about the meaning of an 
experimental laboratory therefore seems necessary, even though not easy. 
Existing definitions, concepts, and experiments provide comfortable modes 
for safe progress. New ideas for laboratory design, among them "simula- 
tion" and applications of computer technology, force the researcher to 
demand answers to whole sets of questions not previously raised in the 
field of education. 

The evaluation-revision concept referred to above forms the basis for 
the meaning of "laboratory" which emerges during this study. Such a 
laboratory must adopt a long-term programmatic approach and guard against 
the fractionation inherent in an unrelated series of small projects, which 
is the orientation that typifies the research activities of many centers 
today. It would also have to take into account such factors as the staff 
training problem, the logistics required to maintain and improve the 
educational system once it is implemented, the strategy required for 
changing the existing education system, and the provision of a built-in 
self-correcting mechanism to assure quality control after the novelty 
of the innovation wears off. It is of course important that the labora- 
tory be user-oriented. If the teacher, the administrator, the parent, 
and the student in schools where new approaches are to be implemented 
have not been involved in the design stage of development, little hope 
can be held for a smooth transition from development to implementation. 

Thir project makes no attempt to conclude that such a laboratory either 
should or should not be supported by federal funds2, since this question 
presumes further progress into laboratory implementation planning than has 
yet been made. (See Section IV. H below.) 

Underlying this projects' analysis of laboratory methods is a basic 
problem: new methods raise questions for which data adequate for the 

techniques are simply not available. A requirement therefore remains 
for explicating a methodology (or plan of attack) which permits an experi- 
mental design appropriate to the nature of the questions. For this 
purpose systems procedures hold great promise. 

Systems strategy means many things to many people, although the systems 
approach is generally described as a defensible set of methods, techniques 
and intellectual tools used to study and improve the achievement of 
objectives in an organization. It should be emphasized that system 



1 See, for example, 3, 11, 17, 22, 26, 37, 48, 52, 75, 81, 105, 109, 
111, 120, 132. [Reference numbers throughout this report are keyed to 
the Bibliography, Appendix A.] 

2 

For governmental roles in related fields see, for example, 3, 8, 

9, 28, 31, 36, 38, 41, 47, 66, 90, 94, 95, 97, 100, 104, 106, 120, 122, 
124, 125, and 135. 
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analysis is significant in that it leads to system design and thence 
to systematic operation. /The_systems strategy is helpful in isolating 
components and in relating organizational subsystems. It is an 
excellent tool; however, it is neither a panacea nor a guaranteed plan. 

! Complex problems, such as those present in an educational environment,^ 
ijiave a way of never being totally resolved. ^ 

i In this study, the systems approach began with a premise that .the 
^educational enterprise achieves its apparent objectives primarily 
^£brough people, rather than through machines. Furthermore, the 
systems approach of industry must be modified if it is to serve school 
enterprises properly, since industrial organizations generally utilize 
rigid barometers to indicate degrees of success and failure — such as 
profit or loss statements, measurement of tangible products, and 
quality assurance standards — while school systems generally operate in 
the absence of such definable criteria. In business, functions can be 
tightly described and policy decisions can be enforced; moreover, 
competition and the discipline of the balance sheet provide elements 
of automatic correction. Basic to the entire concept of systems 
procedures when applied to the educational environment is a belief that 
if the systems concept is to be helpful to educators, it is important;^ > 
recognize that schools exist primarily for students. 

As used here, the systems approach in education is an orderly method of 
attack or a planned strategy for identifying the complex processes 
through which an educational organization operates as it attempts to 
achieve its objectives, with a view toward studying specific subsystem 
interfaces, coordinating subsystem operation, and improving the overall 
educational program through system redesign. The approach is further 
characterized by close attention to the following: 

- Objectives. Priorities . It is important that objectives, or 
trends, or values, can be or have been generally agreed upon. 

It is less important that such objectives, or trends, or values 
have been or can be clearly and unambiguously written down to 
everyone's complete satisfaction, since operational agreements 
are easier and more significant than agreements on phraseology. 

A priority system for implementation of objectives, trends, or 
values in terms of meaningful changes in students is important, 
and helps to clarify statements of objectives. Objectives must be 
realistically attainable. 



^There is a growing body of literature on the use of systems in 
education. The following present good theoretical frameworks and 
descriptions of applications: 3, 5, 6, 10, 23, 25, _»0, 31, 38, 62, 73, 

82, 102, 106, 115, 117, 118, 128, 135. 
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- Information Flow. Accuracy of Feedback . Before clarity can be 
introduced into decision processes, the decision makers must have 
a defensible mechanism for gathering valid, reliable, and 
reasonably comprehensive data on needs, problems, resources, 
accomplishments, and so forth. 

- Alternative Procedures. Interdisciplinary Approach . The analysis 
must always consider the broad spectrum of alternative procedures 
which could be utilized in achieving goals. This process pre- 
supposes that the components in an educational environment can 
best be assessed by a grouping of persons trained in several fields 
and representing all levels of system operation (students, teachers, 
administrators, and researchers, for example). 

- Resource Allocation. Environmental Analysis . Analysis and design 
must identify environmental constraints and opportunities, then 
indicate optimal allocations of resources. 

- Evaluation and Improvement . Operations must be regularly evaluated 
against objectives, with a view toward planning and implementing 
system improvements. 

C. Project Methods 

The system approach described above forms a background to the methodology 
of the present study; however, the study does not carry the method 
through to completion by actual implementation and evaluation of a 
system. The project’s method has been exploratory, conceptual, and 
critical. No experiments have been undertaken, nor has a prototype 
laboratory been built and tested. Instead, expert opinions representing 
many disciplines and backgrounds have been sought out, both inside and 
outside of SDC. Written accounts of experiences and of research with 
multi-media have been examined. [See Appendix A.] Innovative installa- 
tions have been visited. [See Appendix B.] Above all, progress in the 
multi-media "state-of-the-art" has been critically scrutinized from the 
point of view of the educational systems approach. The project s 
guiding themes have been first, to assess the extent to which media have 
been effectively used as resources within a real-world context to meet 
significant educational goals; and second, to define a methodology and 
an organization which could lead to using media as integral parts of a 
greatly improved system of education. 

An exploration of the literature on laboratory research and on the 
implementation of educational innovation did not provide a satisfactory 
basis from which to raise the question of an experimental laboratory. 

For example, there are presently conflicting studies on achievement 
using instructional television, indicating both greater and lesser 
achievement. One may locate research studies to defend instructional 
television or refute its use. The extent to which educational 
technology, learning objectives, student needs and cost-effectiveness 
are related is unclear, since the various interfaces are seldom investi- 
gated. It proved necessary, therefore, to move ahead of the literature 
by exploring significant developments as they were occurring. From 
this view of technological projects, viewed in the context of the 
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preceding description of systems procedures, it was assumed that some 
common processes of design and decision would arise. These kinds of 
information might then provide a foundation for developing the data 
base required by modern exploratory techniques. This process of analysis 
would also help free the thinking of the project staff and permit a fresh 
view of multi-media problems. 

This process of exploration proved its worth in earlier SDC projects in 
education, notably with the Traveling Seminar for Educators [108], 
sponsored by the U. S. Office of Education. Based on this experience 
and on recent change process literature which suggests the importance 
of broad experiencing, the project staff selected a number of locations 
which appeared to offer significant insights concerning the multi-media 
design process [Appendix B]. These sites were visited, meetings were 
held with significant individuals, facilities were toured, and the 
effectiveness of the installation was related to the goals described 
for it. By moving the project staff to the installation it became 
possible to observe the system created by the people. It was felt that ^ 
a conference or large group meeting would have disassociated the project s 
operation from its discussion. Conference reports and speeches do not 
always accurately reflect operational reality; only a close view of what 
is actually happening can produce the kind of analysis required to 
justify additional laboratory expenditures. 

In much of this report, little reference is made to specific media such 
as programmed instruction, language laboratories, motion pictures, 
educational television, and computer assisted instruction. Reference 
to such labels implies that the critical factor in comparing media is the 
mode of stimulus presentation rather than the content of the displayed 
message and the manner in which the student’s performance is analyzed 
and used to determine the sequence of instruction. The usual global 
comparisons (such as ETV versus Motion Pictures to answer categorical 
questions such as which display mode is better) are no longer considered 
useful or relevant to the. task of developing instructional systems, and 
have too often masked the need for all instructional systems to contain 
capabilities for collecting, processing, and monitoring student perfor- 
mance data. In this report, media are usually discussed generically. 
Although the resulting text is at times abstract, wherever possible 
educational problems are used as examples to provide a tangible reference, 
In addition, Appendix B contains a number of concrete illustrations. 

Finally, this project’s method has been interdisciplinary and iterative. 
SDC specialists in educational systems, psychological research, labora- 
tory design, and simulation techniques participated, either as project 
staff members or as project consultants. The ideas and conclusions 
presented have emerged out of a number of preliminary formulations. 
Monthly progress reports were submitted to the Bureau of Research, Office 
of Education, meetings were held, and an Interim Report was submitted 
for review and criticism prior to the preparation of this final report. 
However, it must not be assumed that unanimous agreement of all persons 
concerned was reached on all points. The report represents the 
professional judgment of the SDC project staff, not necessarily of all 
participants or of all reviewers of intermediate statements. 
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II. Background for Feasibility: Current Multi-Media Problems 



This project’s approach in all cases is that techniques should be 
subordinate to problems. Before determining the feasibility of an 
experimental laboratory and the techniques it would use, it is there- 
fore first necessary to specify the types of problems which need to be 
addressed Appendix B of this report contains descriptions of 
representative multi-media applications, with indications of problems 
encountered. The present chapter contains a more abstract account of 
problem areas. However, this discussion of problems is of necessity 
not entirely divorced from consideration of possible solutions. 

Problem analyses and solutions do not exist independently of each 
other, and there is little point in analyzing problems in separation 
from the actual or potential environments out of which resources and 
methods must be extracted in order for solutions to be possible. 

A*> Problem Locations 

A convenient framework for presenting some currently existing multi- 
media problem areas is provided by the Guba-Clark model of the educa- 
tional change process [99, 130]. The stages of educational develop- 
ment have been variously described, but the Guba-Clark model has the 
advantage of dividing the total process of research - innovation - 
implementation into convenient units for analysis. One way of 
determining the need for additional laboratory efforts would be to 
show that a significant step in the developmental process was being 
either omitted or given inadequate attention, and then to show how a 
laboratory could satisfy the requirements. The Guba-Clark model 
includes the following activities: 



Mai or Phase 


Activity 


Purpose 


Research 


Research 


To advance knowledge 


Development 


Gathering operational 
and planning data 


To identify operational 
problems 


Development 


Inventing solutions 
to operating 
problems 


To solve operational 
problems 


Development 


Engineering packages 
and programs for 
educational use 


To operationalize 
solutions 


Development 


Testing and evaluating 
packages or programs 


To assess the effective- 
ness and efficiency of 
the packages or programs 


Diffusion 


Informing target 
systems about 
packages or programs 


To make potential 
adopters aware of the 
packages or programs 



o 
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Major Phase 


Activity 


Purpose 


Diffusion 


Demonstrating the 
effectiveness of the 
packages or programs 


To convince the 
adopter of the 
efficacy of the 
packages or programs 


Diffusion 


Training target 
systems in the use 
of the packages or 
programs 


To develop a level 
of user competence 
with the packages or 
programs 


Diffusion 


Servicing and 
nurturing installed 
innovations 


To complete the 
institutionalization 
of the innovation 



The Guba-Clark model is, within limits, an adequate representation of 
the developmental process. However, the version outlined above should 
be supplemented by explicit recognition of the non-linearity of real 
change. Innovations need not, and very often do not, arise in the 
first instance out of research activities. Innovations may occur by 
chance, or may be invented in response to the existence of comparatively 
isolated and particular problems. Diffusion, modification, evaluation, 
and research then may or may not follow, in any order. 

We are concerned only in part, however, with how educational changes 
in fact take place. Our purpose is to determine how educational changes 
ought to take place, in order to produce beneficial results most 
effectively and rapidly. For this purpose, the following diagram 1 of 
the process of developing a complex man-machine system (Figure 1) 
is a helpful supplement to the Guba-Clark model. It represents an 
expansion of the development segment of a controlled educational change 
process, indicating the critical nature of interactions and feedback 
among the elements of the process, and emphasing the necessity for 
planning, management, and evaluation during the entire developmental 
effort . 

Research with educational media may, for example, suggest advantages 
to be derived from computer applications. It is a long distance, how- 
ever, from such a suggestion to successful installation and operation 
in actual educational situations. As Figure 1 indicates, successive 
stages of planning, design, description, specification, and production 
must be embarked upon, with adequate attention given both to equipment 
(hardware) and to lesson materials and human procedures (software) . All 
adjacent stages interact (that is, imply mutually induced modifications) . 
And the final operational stage reacts back upon all previous stages, 
since actual operation provides the ultimate basis for evaluation and 
improvement. 



Adapted from SDC TM-864, "Software Design and Implementation," 



1962. 
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If education is thus regarded as a system, with media as one of its 
aspects or subsystems, it clearly follows that "multi-media problems" 
neither exist nor can be solved in isolation from the total system. 

In addition, the system’s environment must also be fully considered, 
since the environment is the primary location of both system constraints 
and system resources. The environment includes both the explicit and 
the implicit political, economic, and social forces which impinge on 
the system. It causes system problems, and it can also provide problem 
solutions. For Instance, suppose a program of training, emphasizing 
child growth and development, were provided for all expectant mothers. 
Such an alternative might produce greater results than similar 
expenditures for additional pre-schools, for nursery-level educational 
television, or for some other scheme. This essentially educational 
program, however, while affecting the K-l? input and thus the educa- 
tional system, would require changes in areas beyond the limits of 
the ordinary school system, as well as adaptive changes within it. 

In accordance with this point of view, "multi-media problems" should 
not be expected to be found in any single area marked out within a 
model of change or development. A laboratory to address these problems 
should not concern itself with research activities alone, or planning 
and design alone, or operation and evaluation alone. Each of these 
activities is one tool which can be used, along with the others, toward 
solving educational problems. None of them should be undertaken as an 
end in itself. 

The remainder of this chapter reviews multi-media problem areas at 
a somewhat lower level of generality. (For specific examples, see 
Appendix B.) Current multi-media practices and research are briefly 
described, as a basis for identifying several critical needs which at 
present are very far from being met. The fundamental overall need 
(problem) is for systematic development, evaluation, and demonstration 
of multi-media uses in real situations for actual educational purposes. 
In comparison with what is required, present efforts are partial, 
sporadic, fragmented, and inconclusive. 

B. Present Multi-Media Practices and Research 



Multi-media as they are being used in schools and colleges are generally 
proving to be educationally ineffective. They are not accomplishing the 
educational objectives intended; they have not been found to be cost- 
effective; and they have raised a number of problems which were not 
anticipated. This condition persists despite the tremendous investment 
in mediated devices, the funding of considerable research, and the close 
analysis of a number of multi-media situations. 1 



^There are innumerable analyses which lead to this conclusion, 
among the better ones: 5, 6, 13, 18, 20, 31, 37, 39, 40, 53, 55, 

60, 69, 70, 71, 77, 79, 85, 99, 107, and 114. But see also the 
positive directions indicated by 120 and 137, among others. See also 
Appendix B, this report. 
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Multi-media have been used primarily to attack two problems which are 
especially relevant at the administrative rather than at the learning 
level. The first problem which multi-media were asked to solve was 
the presentation of material to a larger number of students than could 
be accomplished by a single teacher. This problem involved the 
reproduction, transmission, storage, and retrieval of information, 
usually in the same format as it would be presented by the live teacher. 
The second problem addressed by multi-media was the individualization 
of instruction through presentation of a controlled experience to a 
single student. Here the essential problems included the particular 
needs of the student and the adjustment of the learning experience to 
these needs. 

In both of these applications, a number of essential questions were 
left unanswered. These concerned the appropriateness of the experiences 
provided, the relation of multi-media presentations to other activities 
in the learning environment, and their long-range effects on the pattern 
of education. The difficulty with these uses of multi-media stems from 
the fact that media were implemented within a system which was designed 
prior to their existence, and only vary minor attention was devoted to 
the implications of their use for the rest of the system. While this 
difficulty may be expressed in terms of "too narrow a definition of the 
problem situation," other concerns are even more basic than this expres- 
sion implies. The existence of multi-media (or any of the sophisti- 
cated media singly) makes possible the accomplishment of worthwhile 
educational goals otherwise not attainable, but the effective use of 
multi-media may require a complete reorganization of the present system 
of education. Multi-media and telemedia applications should be analyzed 
and planned within the context of potential radical changes in education 

While the application of technology to education, through the develop- 
ment of multi-media, telemedia, and other devices, may in the long run 
lead to a restructuring of American education, educators may still ^ 
isolate significant uses of media in present educational environments. 
The following uses, and the problems associated with them, represent 
basic areas of multi-media discussion within this report: 

- Improving the relevance and rapidity of information transmission, 
storage, and retrieval. 

- Improving the appropriateness of media for the communication of a 
message. 

- The recording and immediately playing back of both audio and 
visual records of student behavior and/or teacher behavior for 
purposes of review, analysis, and control. 



^Actual and potential uses of multi-media resources aie usually 
described in terms of fulfilling some function within the present 
environment. Good catalogs include for multi-media, 37, 33 » Jr 0; 
for computers, 17, 40, 49, 111; for television, 35, 87, , > 

for information system, 16, 15, 26, 117; in general, 19, 23, 31, 33, 

94. 
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- Time saving and other operational efficiencies. 

- Improving administrative procedures for the management of instruction, 
including the selection of appropriate learning tasks and immediate 
evaluation feedback, thus permitting individual adjustments in the 
processes of instruction. 

- Overcoming the limitations of the native environment through the use 
of simulated environments and through the mediated presentation of 
other environments. 

- Generating detailed analyses of the learning situation, forms of 
presentation, adjustments in the situation, and needs for information 
and redesign. 

Considerable research in the multi-media field has concentrated on 
throe important areas: tests of the effectiveness of media, tests of 

the superior effectiveness of one medium over another, and tests of 
the reactions of receivers to mediated messages [12] . This research 
either has had limited interest in the effects of environmental factors 
or has chosen to ignore them; as such, the research has not been 
generally applicable in useful ways to real learning situations in the 
schools. Other research efforts and studies have dealt separately with 
the support of media — physical facilities, light and noise levels, 
seating, and related concerns — in an attempt to improve the conditions 
in which media are used. Some attention has been devoted to adminis— 
trative and management problems relating to multi-media implementation 
and use. A considerable and growing body of literature explores the 
nature of learning processes and interrelates multi-media, principles 
of learning, and the conceptual organization of subject matter. !_kl- 
though a tremendous amount of money and effort is being devoted to 
multi— media research, something seems to be missing. (All of this 
research and development has failed to result in any major change 
either in the use of multi-media within the schools or in the tasks 
of education. Whis conclusion provokes two important questions. First, 
should major new efforts be undertaken in the development of larger 
technological systems, such as ITV, before something more is known of 
their values? Second, what kinds of activities are needed to discover 
these values and put them into effect? 

C. Some Critical Needs for Analysis. Development, and Diffusion 

In order to make fruitful use of the tremendous educational potential 
offered by new media, new kinds of investigation and development are 
needed. These range from conceptual analyses through test and evalua- 
tion to systematic installation and operation in actual schools. The 
following paragraphs discuss critical needs for new efforts. 

L 1. Analysis of the Total Multi-Media Learning Situation . Exten- 
sive effort should be devoted to defining the elements of the multi- 
media learning situation and their significant interrelationships, 
since a fundamental weakness in present research efforts is the limited 
conceptualization of factors affecting media use. This situation can 
be improved only through deliberate efforts to analyze the environments 
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in which multi-media operate, to determine the significant factors 
within this environment, and to interrelate these factors in a total 
design schema. This statement suggests the application of various 
forms cf modeling, simulation, gaming, and other modern techniques 
of research. 

This consideration implies several other areas of concern which are 
basic to these analyses and which continue to remain enigmatic problems 
of education. Multi-media can be effective for the accomplishment of 
particular goals of learning, but the relation of these goals to the 
operation of the schools has so far defied analysis. The constraints 
imposed upon education by the nature of the institution, by the purposes 
of schooling other than for education, and by available funds and 
political energies will affect the multi-media environment. Analyses 
of learning situations, therefore, must include not only the immediate 
environment of the learner but also the myriad of forces which impinge 
upon the act of learning. 

Research efforts, therefore, should be directed toward an understanding 
of the total learning situation and the interrelation of all the forces 
impinging upon it. Special encouragement should be given to the develop- 
ment of models and simulations of alternative learning conditions 
employing both mediated and non-mediated techniques. 

2. Conceptualizing Education as a Man-Machine System . The majority 
of multi-media uses so far have been casual; that is, teachers have not 
really depended on media to perform specified functions within the 
classroom. If the movie projector fails to operate, for instance, the 
teacher presents the same information by the lecture method. In fact, 
it can be substantiated that a good deal of media usage arises from the 
desires of teachers to ease their tasks through using prepared material, 
"filling in" when a regular teacher is missing, or through serious but 
limited attempts at "enrichment." This last use implies that the media 
are at best something extra — not a natural part — of the learning environ- 
ment. 

A man-machine system implies the interrelationship and joint functioning ^ 
( of man and machine for the accomplishment of particular goals. J Within 
^the system, tasks are assigned to men and to machines, and each is held 
responsible for the achievement of its function. In those systems 
where this concept has been applied, certain tasks have been designated 
as appropriate for machines, while others have been reserved for men. 

As a total system develops, however, machines are designed to relate to 
men, and men are trained to operate and interact effectively with machines 
Since both men and machines "fail" on occasion, back-up systems are design 
ed to cover emergencies, but full operation is achieved only when men 
and machines are functioning properly. 

The application of man-machine systems concepts to educational design 
and planning holds great promise. Indeed, such planning is requisite for 
the development of adequate mediated systems. Unfortunately, insufficient 
attention has been devoted to this area, in part because of exaggerated 



fears about the dehumanization of education by machines.'*' The planner 
attempting to employ such techniques is usually greeted with a barrage 
of ethical and moral quandaries concerning the nature of the individual, 
and, while these arguments are extremely signifiicant , they are usually 
presented in a form which impedes progress rather than suggests alter- 
natives. The questions of appropriate functions for a man, or appropri- 
ate responsibilities for a machine, especially within an educational 
environment, are extremely real and extremely difficult, but their 
explication and resolution may be a necessary preliminary to the develop- 
ment of effective media systems. 

, . 3. Effects on Individual Learners . An increasing use of telemedia 

and multi-mediated learning modes can produce adverse effects on indi- 
vidual learners through preventing adequate and appropriate inter-group 
communication, teacher-student communication, or effective release of 
the creative potential. These conditions would be most in evidence as 
content and presentation are highly structured, even though this 
structure may include special considerations for individual learning 
styles . 

There is little evidence to substantiate the loss of humanizing factors 
in situations which involve the use of multi-media or telemedia unless 
these uses were designed without adequate attention to the needs of the 
learner as a person. (As much can be said for non-mediated systems) . 

The increasing sophistication of technological systems and the increas- 
ing dependence on technological components in the learning environment, 
however, require that deliberate attention be devoted to insuring that 
adequate opportunities for self expression, human interaction, and 
personal involvement be included in the total design. 

Multi-media applications within classrooms and other learning environ- 
ments should be examined from the standpoint of goals other than those 
connected with the presentation of learning of a formal body of know- 
ledge. Areas of consideration should include: 

- Enhancement of individual integrity and self-actualization. 

- Effects on social values and interactions. 

- Effective application of new knowledge within group situations. 

- Development of appropriate life styles and roles including a 
high degree of flexibility. 

- Development of initative in learning and self-direction in the 
selection of appropriate activities and tasks. 



There is some indication that this situation may be improving, but 
planning has still not generally assumed a realistic attitude in this 
area. See references 46, 79, 85. 
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- Development of appropriate individual control and evaluation 
devices for the reception and integration of information. 

Another critical issue raised by the development of learning systems, 
and by most multi-media efforts, concerns the degree to which the 
individual should be the creator of his learning situation rather than 
only the receiver of previously planned experiences. Throughout the 
ages, the classroom has been an arena for the presentation of information 
from a single source to multiple receivers. Any feedback which occurred 
was controlled by the single source of information, the teacher. Success 
was determined by criteria selected by the teacher as appropriate and 
respectable for the information being presented. While the progressive 
movement and the "discovery" and "problem solving" methods have produced 
some change, this pattern still remains dominant in the schools. Media 
can help to change this pattern if the significant element in classroom 
interaction is no longer simply the acquisition of information (or even 
the solution of specific problems), but too little attention has yet 
been devoted to explicating methods of using multi-media for active 
rather than passive learning. 

Interactive Learning. Almost all present media uses have been 
based on traditional concepts of the teacher-learner relationship. It 
is unfortunate that there has been a consistent failure of multi-media 
(including telemedia) applications to increase the range of interactions 
within the learning situation. The assumption appears to continue that 
if material is presented more clearly, more intensely, in more dimensions, 
or more kinesthetically , learning will increase. Every current 
example of multi-media use is based on the presenting of information 
from a single source (or group of mediated sources acting in concert) to 
one or a group of students. These students may or may not respond to 
the source. If they respond, they may do so individually or as a group, 
through a sophisticated technological system or by routine procedures. 
Within this context the material may have been organized quite expertly, 
and the latest developments in learning theory and psychology will be in 
evidence. In fact, some of the multi-media classrooms epitomize this 
style of learning. 

However, little evidence exists of a change in the teacher’s role as a 
result of mediated environments. Some teachers may revise their time 
schedules as a result of media, but the list of functions remains the 
same. No mediated classroom so far has made provision for the comfort- 
able interaction of groups of students, or for direct student-to-student 
interaction within the context of the subject matter. Yet it appears 
that the true vitality of the new media rests on the ability to promote 
just such new and exciting modes of learning interaction both among 
individual students and other persons, and between the individual student 
and the technological system. 

The great danger for telemedia and other forms of mediated instruction 
will be their failure to change the organization of learning processes. 

It appears clear, however, from the advanced literature, especially 
dealing with media experimentation in other areas, that important 
breakthroughs can be achieved. The new media are capable of increasing 
the range of interactions in the learning situation and of supplementing 
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the traditional teacher-to-student-to-teacher classroom pattern. Re- 
search and development in this area should receive major support and 
encouragement . 

5. Effects on School Organization . In the past, schools (like 
all institutions) have grown more in response to random pressures and 
opportunities than as a result of systematic planning, development, and 
improvement. Today, it is becoming possible for this state of affairs 
to change. It is true that theoretical dangers exist from over-planning 
and over-centralization of control of the schools. These dangers seem 
remote, however, as long as fundamental commitments to local control, to 
experimentation, to rigorous analysis and evaluation, and to competitive 
information exchange are preserved. The opposite dangers — of no planning, 
no systematic evaluation, and no coherent steps toward improvement — are 
much more likely and more threatening. 

There is a clear and immediate necessity for American schools to be 
significantly improved. The new media can play a major role in this 
process. Questions about the shape and functions of the schools of the 
future become unavoidable. Even if schools as we know them do not some 
day cease to exist, they will surely undergo profound changes. Should 
these changes be unplanned? 

For example, as telemedia networks develop, radical changes in the 
organization and operation of the schools will become advisable. Some 
areas of local decision may be transferred to central authority, simply 
because by their nature technological systems provide services beyond 
the needs of individual districts and, in turn, require resources beyond 
the local district. The organizational and political problems suggested 
by these statements indicate the need for development of implementation 
models describing predictable changes in control, new trends in the 
allocation of resources, and positive and negative socio-political 
factors. 

Telemedia, and to some extent all mediated devices, require changes in 
learning spaces. The information function of education need not be 
limited to the classroom, and in fact reception of ITV in the home may 
be desirable under certain conditions. If this example is extended, the 
school as presently conceived loses its primary function, and its purposes 
and uses must be rethought. This concern is, admittedly, one of the 
distant future, but telemedia make possible the redesign not only of the 
school but of its support elements, its relationship with the home and 
other community institutions, and its relation to the total needs of 
the population it serves. 

6. Allocation of Critical Resources . Decisions about the use of 
milti-media and telemedia technologies for education should be based 
on careful and continuing analysis of the comparative effectiveness of 
alternative allocations of available resources. These include not just 
the costs of media systems and their direct support, but implications 
for teacher time (in relation to costs of specific tasks and alternative 
staffing patterns), student time (including length of school day and 
year, early graduation, and alternative uses of time) institutional 
resources (maximum feasible use of buildings, library collections, 
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technologies, etc.), and other resources outside of the educational 
sphere but available to the community. 

Models of resource allocation and conservation should be developed so 
that adequate information can b*= available for trade-off decisions. 

The new technologies may indeed make possible vast savings in student 
time, perhaps even the completion of twelve years of schooling in 
eight, but what will be the result of such a change in the economic 
and social patterns of the lives of youths? Other important trade-off 
areas are related to the costs of teacher preparation, procurement, 
and employment related to the use of technology to perform some 
teacher functions. 

Education has been traditionally a labor-intensive industry, but the 
new technological breakthroughs indicate that this situation may 
change. Indeed, as the costs of providing adequate labor resources 
increase, pressure will be felt for the exploration of such changes. 
The literature of economics traces the effects on some industries of 
a change from labor-intensive to capital-intensive (and/or technologi- 
cally oriented) , but there is little such study in the field of 
education. This area should receive substantial consideration as 
sophisticated technological systems are developed. 

7. Real World Problems of Classroom Teachers . For the foresee- 
able future, the effective use of media will continue to depend upon 
the classroom teacher and the support the classroom teacher receives 
within the building. The following areas are in need of particular 
attention: 



Media Training for Teachers 

Present teacher preparation programs offer only limited experiences for 
prospective teachers in the use of the new media, and in-service efforts 
do not keep pace with faculty needs. Several excellent models for such 
training have been developed and are in use, but the magnitude of the 
problem requires increased attention to the specific training require- 
ments of teachers who will find themselves working with telemedia. 

A training segment should be an integral part of any media implementa- 
tion plan. As well, efforts should be directed towards the development 
of on-site, self-teaching, mediated training programs for in-service 
use . 



Criteria for the Selection and Use of Media 

Models for the selection and use of media should be developed to guide 
classroom teachers in the effective use of multi-media under varying 
conditions. These models should include not only the description of 
appropriate content and presentation, but also some indications of the 
implications of their use, such as scheduling problems, space 
problems, etc. This information should be presented in such a way that 
teachers will be able to make reasonable decisions concerning the uses 
and advantages of the various media. 
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Resistance to the Use of Multi-Media and Telemedia 



Teachers resist the use of multi-media and telemedia because of (1) 
generic fears of the unfamiliar, (2) problems related to procurement, 
operation, and classroom control, and (3) inadequate understanding of 
media functions and potentials. Attention should be directed towards 
understanding this resistance and towards building implementation 
models which realistically deal with it. 



Multi-Media and Telemedia Materials 

There is an extreme overproduction of low-grade material for use with 
media and a lack of adequate critical information to guide selection. 

This fact, coupled with distribution problems, causes many teachers 
to reject the use of media. Much of this difficulty can be traced to 
the management of media at the local school level, but some remains 
the responsibility of the producers and designers of packaged products. 
The production and distribution of superior materials should be 
encouraged and supported at both the development and implementation 
stages. In addition to ancillary materials, entire multi-media courses 
should be developed. 

8. Evaluation and Demonstration . Virtually every conceivable form 
and combination of media use can be discovered somewhere within the 
United States. Some innovations have become popular and are replicated 
frequently, but others represent unique developments, usually in the 
research and early development stages. One difficulty with these early 
efforts at implementation results from the restricted funding which can 
support them. They are therefore limited in their effects, while only 
insufficient information is recorded about the problems of implementa- 
tion and use. Later interested adapters cannot find adequate models 
upon which to base decisions about a media plan, and their developments 
must begin from scratch. 

Present methods do not disseminate adequate information about experiences 
with mediated systems, and only infrequently can a school district 
determine what implementation problems will be faced if it chooses to 
implement a major technological innovation. A number of efforts, 
especially at the Federal level, have been initiated to obviate this 
difficulty, but they have been aimed only at pieces of the problem. What 
is required, it would appear, is the allocation of sufficient resources 
to the development and implementation of significant media innovations 
in representative school districts. These demonstration districts 
would be provided with the necessary resources to evaluate and record 
their experiences in mounting an innovation, and would continue as 
research and development centers for test and demonstration. Close 
ties with universities and production agencies would be required, and 
special considerations would be available for teachers and other 
personnel. 
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HI. Feasibility of Alternative Simulation Laboratory Approaches 

to Multi-Media Problems 



A. Types of Laboratories 

The previous chapter indicates a great range of problems existing with 

respect to the effective use of educational media. The following six 

types of laboratories (or laboratory-like organizations) can be 

distinguished as being able to deal in various ways with these problems: 

- Planning and Policy Institutes (studies and predictions of future 
needs; recommendations of goals and methods) 

- Basic Research Laboratories (studies to discover and verify general 
principles) 

- Component Development Laboratories (activities aimed at discovering, 
testing, perfecting, and demonstrating equipment, materials, and 
procedures in comparative isolation from their final intended 
environment) 

- Simulation Laboratories (intermediate between component laboratories 
and laboratory schools; activities take place in quasi-total environ- 
ments in which as many factors as possible are either actually 
present or represented by more or less realistic simulations) 

- Laboratory Schools (activities aimed at discovering, testing, perfect- 
ing, and demonstrating equipment, materials, and procedures in actual 
educational situations; single classes, schools, or groups of schools) 

- Comprehensive Development Laboratories (activities aimed at continuously 
planning, supervising, evaluating, and supporting systematic educational 
improvements; aspects of the five preceding laboratory types are 
coherently combined) 



Of these six types of laboratories, at least three already exist and 
function more or less effectively. These are basic research laboratories, 
component development laboratories, and laboratory schools. In addition, 
educational planning and policy activities are fairly widespread. The 
two educational laboratory types which do not yet clearly exist, and 
which promise needed assistance in solving multi-media problems, are 
simulation laboratories and comprehensive development laboratories. 

The remainder of this chapter discusses the feasibility of simulation 
laboratories. The following chapter will consider the comprehensive 
development laboratory. 



B. Simulation Laboratory Techniques and Pu rposes, 

Simulation is a general term meaning representation by analogy. There 
are three forms of simulation which can be conveniently distinguished. 
The first is physical, represented by flight simulations, in which a 
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physical mock-up replaces a more expensive or less accessible original. 

The second form is symbolic, including verbal representations, drawings, 
and mathematical models. The third form of simulation is "digital," 
a general term for developments falling within the field of computeriz- 
ed simulation. Initial experimentation with digital simulation techniques 
raised considerable enthusiasm for their use in complicated research 
problems. Their success, however, has often been limited in relation 
to their costs. 

Simulated objects and situations are used as substitutes for reality for 
a great variety of reasons. In all cases, however, the simulation must 
have some distinct advantage over reality, such as lower cost, greater 
controllability of variables or environment, convenience or possibility 
of accessibility and use, avoidance of danger to people or equipment, 
replicability, or ease of construction and operation. Sackman [118] 
distinguishes two forms of simulation, operations simulation and 
symbolic simulation . "By operations simulation is meant the simulation 
of operations in an information-processing system wherein people and 
computers are involved, and by symbolic simulation is meant the simula- 
tion of an information processing system through symbolic or digital 
representation of logical relationships, wherein real people are not 
involved but computers generally are involved." In either case simula- 
tion is basically "the representation of experimental objects, and the 
way the experiment is conducted to produce experimental results." 

An effective simulation must portray an accurate description of the 
environment within a consistent degree of specification. Robert Fitz- 
patrick [44] lists these general classes of systems aspects which should 
be included: 

1. Equipment components : These may be components in the usual 

sense, or they may be sub-systems, or even whole systems, so long as 
they consist only of "hardware." 

2. Personnel : Included here are all individuals associated with 
the system, whether managers, operators or maintenance personnel. Also 
included are their job responsibilities organized as necessary according 
to tasks, shifts, etc. Not included is the organizational and social 
environment, as described in 3, below. 

3. Organization : This category is intended to embrace both formal 

and informal organization of the work force, social status and interactions 
of individuals and groups, degree of isolation of individuals and groups, 
and communication procedures insofar as these are not specified in 
individual job descriptions. 

4. Input data : Information, energy, and materials explicitly 

recognized as those which provide the necessary and sufficient basis, or 
"raw material," for system operation. 

5. Output data : Information, energy, and materials explicitly 

recognized as those the system is designed to produce. Adequacy of the 
system is measured by the appropriateness, timeliness, accuracy, and 
comprehensiveness of the output data. 
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6. Systems procedures and processes : These are the rules by which 

the system operates, other than those specifying organization or job 
responsibilities of the personnel. Examples are computer programs, 
policies, and operating procedures. 

7. Environment : Included here is everything not covered by the 

other items: the physical, political, economic, and other contexts in 

which the system is embedded, but which are not recognized as parts of 
the system. 



Fitzpatrick expresses clearly the basic dilemma in the application of 
simulation techniques: 

"If simulation is completely representative so that perfect 
validity is obtained, the simulated system is just like the 
real system and hence equally difficult to control experi- 
mentally. Experimental control is best obtained when the 
number of operating variables is reduced sharply and each 
variable is defined narrowly, as in a laboratory experiment. 

The probable error of an estimate of a parameter resulting 
from a rigorous laboratory experiment is as low as possible. 

The relevance of the parameter to real life is often obscure, 
but the value of the parameter is precisely determined. This 
then, is the dilemma of simulation: High representativeness 

is thought to be required for high validity, so that what- 
ever statements are made can be considered relevant to the 
real system. But high representativeness is achieved 
normally at the expense of control, so that the statements 
in that case tend to be approximations only. The basic 
problem in simulation is to compromise between the goals of 
validity and precision." 

A discussion follows of the advantages of simulation techniques over 
"real-world" techniques, as well as of constraints on the uses of 
simulation, with particular reference to the various phases of the 
Guba-Clark educational change model. Each step on the model is 
discussed in terms of four aspects: 

- Present types of real world activity 

- Types of potential simulations 

- Other research techniques 

- Constraints on the use of simulation 
Future Planning 

Future planning involves the determination of needs, the description 
of alternative futures, and planning for changes in present institu- 
tional arrangements. Real world activities in this area are confined 
to observations, to somewhat impractical long views which are limited 
by the effect of operational environments, and by the impossibility of 
abstracting information for prediction. Simulations, therefore, are 




24 



especially useful in this area, including the techniques of scenario 
writing, Delphi investigations, planning games, factor analyses (and 
predictions based on various combinations) , computerized mathematical 
simulations, fictionalized predictions, and the analysis of alternative 
strategies. 

The use of classical research techniques does not appear feasible for 
this type of activity. It should be recognized, though, that simula- 
tions are constrained by the present orientation toward the tradition 
of future planning, the present application of techniques, the lack 
of organized data bases for use in such planning, the absence of 
effective models, and failure to explicate alternative environments. 



Research 



The research potentialities of simulation techniques are covered at 
length in the later sections of this chapter. An additional research- 
related use of simulation is its application to research planning. 
"Real-life" activities of relevance include observations and analyses 
of "action research" for situational improvement, and evaluative 
studies of the adequacy of present research practices. Simulations 
would be useful in this area for the development of paradigms of needed 
research and predictions of costs and developments. Simulated models 
of alternative consequences of research expenditures and/or break- 
throughs could be developed, along with scenarios of the implications 
of R&D on extant environments. 

Classic research of all varieties has usually been used for narrow 
gauge examination and evaluation of the parts of research. The use of 
simulation would be constrained by the lack of statements of effects 
anticipated by research and the lack of a data base on R&D procedures 
in education. As well, there has been no determination of critical 
resources. Moreover, the continuing inability of education as an 
institution to modify its environment significantly suggests the need 
for re-evaluation of the goals of simulation. 



Development (Invention and Engineering) 

In this first sense of the word, development involves the inventing of 
solutions and the engineering of packages and programs. Present real 
world activities include invention and engineering limited to cut and 
fit in present situations, a fact which often warps the context by 
over-developing a single aspect (i.e., hardware ahead of software); 
or it may involve rearrangements of extant elements to achieve 
greater efficiency and reduce the need for invention. Simulations could 
perform a number of services in this area, including a prediction of 
needed inventions and alternative inventions, abstracts of the effects 
of invention on the real world, or needed reorganizing of the real 
world. 

Classical research has provided for the testing of parts under care- 
fully controlled conditions and analyses of relevant factors for 
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simulations. Comparative and reductive analyses are valuable. Simula- 
tions, on the other hand, are constrained by a lack of sufficient 
information on the relevant factors in the real situation, a lack of 
sufficient exploration of some techniques which could be presently 
used, and failure to undertake predictive studies of needed inventions 
(partially because education relies too heavily on derivation from 
other fields). 



Development (Testing and Evaluation) 

In this second sense of the word, development involves the testing and 
evaluating of packages or programs. Present activities include the use 
of the cut and fit method as constrained by the limits of the real- 
world, and the operation and assessment of experimental schools, still 
limited by a lack of predictive information and by building "from 
scratch" from plans developed earlier by simulated or classic methods. 
Simulations would be useful in this area of development for developing 
models of the total test environment for symbolic, prediction-yielding 
trials of alternative organizations of development. They could also 
be used to evaluate the effects of alternatives in simulated environ- 
ments representing alternative real worlds. 

Classical research studies in these areas have included factor studies 
and comparative studies, both of which are usually less valuable at 
this point. Simulations are constrained by the present lack of 
attention to models of educational realities and to creation of 
simulated environments. 



Diffusion (Demonstration) 

Diffusion includes informing and demonstrating, which in the real-world 
under present, conditions would continue to reflect present strengths 
and weaknesses. Simulations could make effective contributions by 
reflecting specific situations, to include potential alterations in 
both situations and environments, indicating the feasibility of 
considering change. The implications of adaptions might be predicted 
over long periods of time. Scenarios, role playing, and Delphi 
techniques could elicit opinion and support for needed change. Portable 
walkthroughs and physical (operational) models could allow for try-outs. 

Classical research presents only the possibility for the usual diffusion 
of its findings through the technical and professional literature. 
Simulation has been constrained in this area through a failure to think 
of alterations in the environments as an effective alternative to 
problem solution (through removing the problem by changing the 
environment) . 
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Diffusion (Training) 



Diffusion also includes training and upgrading, including the present 
real-world activities of seminars, institutes, and college programs. 
Simulations might aid these by the use of mock-ups, trainers, walk- 
throughs, role playing, scenarios, and computer heuristic and problem 
solving techniques which can provide for simulating the total educa- 
tional environment, teacher behavior and reaction (interaction analysis), 
and practice in use and creative applications. 

Classical research is not usually intended for active training, but 
simulation requires major development efforts to redesign attitudes 
about professional functions and the modes of training in education. 

This will take time and financial support. 



Diffusion (Nurturing) 

The third aspect of diffusion involves the servicing and nurturing of 
installed innovations. This is primarily a real world area to which 
neither research nor simulation can contribute uniquely, except by 
extensions of activities already described. Improvements, however, 
could be achieved through the uses of instructional management systems, 
predeveloped devices for "what if" predictions, and controls on the 
selection of experiences. 



C. Experience with Simulation Laboratories for Education 

Before continuing a general discussion of simulation laboratories and 
their uses, it is desirable to give a more concrete idea of simulation 
possibilities by referring to some actual examples. Appendix C of this 
report presents brief descriptions of a variety of actual simulations 
outside of the field of education, as a general background for the 
reader not familiar with applications of simulation techniques. The 
following paragraphs describe two applications of simulation laboratory 
techniques to education. 

An example of a rather complex educational simulation effort was SDC's 
"automated classroom of the future". Designated CLASS — for Computer- 
Based Laboratory for Automated School Systems — this experimental 
operation was established within a general purpose simulation facility, 
the Systems Simulation Research Laboratory. It contained both a large 
computer and closed-circuit television; students’ desks equipped with 
audio devices, film viewers, and response devices; special purpose 
teacher display equipment; and a computer-assisted counseling facility. 
Using students from local school districts, a research team investigated 
problems of programmed learning and automated assistance to education, 
with the long-range goal of developing more effective means of 
instructional support. 

In the CLASS system, each student had a manually operated film viewer 
containing 2,000 frames of instructional material. In addition, he 
had a response device, linked to the computer, which indicated the 
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sequence of slides to be seen by the student. This enabled the student 
to respond to questions and presented knowledge of results to the 
student in the form of a coded light. The computer maintained perform- 
ance records for all students, supplying these records to the teacher, 
counselor, or administrator. 

Provision was made for two teachers in CLASS. Each teacher had console 
facilities which made it possible for him to call up computer-generated 
displays showing the current progress of any student or group of 
students. Automatic alarm lights alerted the teacher to students who 
were performing unsuccessfully in any lesson. CLA.SS also permitted 
instruction through media such as television, films, slides, and 
conventional lecture and textbook methods, permitting instruction in 
either the individual or group mode. 

A second educational application within the Systems Simulation Research 
Laboratory involved research activities designed to find new solutions 
for implementing instructional media through analysis and simulation of 
school organizations. The intent in general was to suggest new 
combinations of media and personnel that would accomplish the tasks of 
education more efficiently and more effectively. A systems analysis 
was undertaken of five high schools involved in the use of new educa- 
tional techniques. This analysis formed the basis for designing the 
simulation models used. The simulation vehicle, essentially a detailed 
dynamic model of a real or proposed school organization, was designed 
to meet the following specifications: 

- The capability of building dynamic models of real or proposed high 
schools . 

- The capability of flexibly modifying models to represent different 
design configurations. 

- Detailed recording of the events that take place during the 
simulation. 

- The production of detailed output data that reflect the effects 
of various design changes within the model. 

- The simulation of events occurring in relation to time. 

The simulation vehicle was constructed so that any high school could 
be described in terms of school characteristics (resources, organiza- 
tion, procedures, and so forth) and of student characteristics that 
bear on the school's instructional plan — for example: 

- The curriculum and its organization. 

- The spatial arrangement of the instructional area. 

- Resources such as programmed learning materials, teaching machines, 
equipment, teachers, counselors. 

- Procedures for channeling students through the counseling system. 
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- Procedures for admitting and terminating students. 



- Procedures for relating to exernal agencies directly concerned 
with helping students. 

- Information-processing procedures. 

- Characteristics of students that relate to instructional and 
counseling processes. 

- Decision-making procedures. 

Experiments undertaken in the area of school organization included: 

- Analysis of a continuous progress course in algebra. 

- Design and simulation of a school in which students schedule 
themselves under a continuous progress system. 

- Studies of the information processing needs within schools. 

- An analysis and simulation of an English course. 

- The simulation of a flexible school. 

"The procedure followed in the development of the modeling 
study was to work back and forth between simulated data and data 
obtained from the real environment. In this manner, the model 
served the function of a theoretical model. It represented an 
explicit statement of the modelers' understanding of what takes 
place in the environment. The computer simulation technique 
both permits and encourages more detailed and dynamic expli- 
cation of the experimenters' understanding. . .The real test 
of validity [occurs] when the experimenter can change the 
organization within the model and obtain predicted effects that 
later are substantiated by the same sequence of events in the 
real environment.” [ 26 ] 

These early experiments with the use of simulation for the study of 
educational problems were effective in providing new directions for 
educational research and planning. Most of the individual experiments 
undertaken, however, were limited in scope or in the number of variables 
included within the simulation. This deliberate simplification proved 
useful for projecting the consequences of changes in certain aspects 
of the system, but there continued to be a need for developing in- 
creasingly broad analyses. In addition, costs were exceedingly high. 
Even though the simulations took place in a laboratory also used for 
several other types of experiments, a great deal of expense was 
involved both in developing computer programs and special materials 
for the educational work, and in preparing for, conducting, and analyz- 
ing experimental sessions. 

As a result of such experiences, with costs comparatively quite high 
and effectiveness rather low, enthusiasm for complex computer-based 
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educational simulations has declined. It should not be concluded, 
however, that simulation techniques ought to be entirely abandoned. 
Instead, they should be examineu more thoroughly to determine 
possibilities for reducing their costs and/or increasing their 
effectiveness. The discussion which follows is a contribution in 
this direction. 



D. Simulation Laboratory Sub-Types 

The range of possible physical laboratory facilities for the simula- 
tion efforts under consideration extends from conventional, manually 
operated facilities to the most advanced computer-based, time-sharo.d 
center. This range can be characterized best by a brief description 
of four general types of facilities: manual, special purpose, and t-70 

types of computer-based laboratories — dedicated and time-shared. 

SDC has had experience with and an opportunity to evaluate each of these 
four types of laboratory facility. Evaluation is offered in terms of 
costs, measurement potential, and adaptability. 

1. Manual Laboratories . The simplest type of manual (that is, 
non-automated) facility consists of little more than rooms, furniture, 
paper, pencils, and people. From this rudimentary base refinements 
are introduced by subdividing rooms into subject booths, concealing 
observers with one-way glass, and augmenting data collection through 
the use of mechanical recording equipment — tape recorder, movie camera, 
TV tape, etc. In general, the manual laboratory consists of a 
collection of standard, off-the-shelf equipment primarily controlled 
and operated by people and configured to meet specific research goals. 

Costs: The manual laboratory requires a relatively small initial 

investment. Its operational costs are also comparatively small, since 
production and implementation do not require large numbers of technical 
personnel. Maintenance efforts (restricted to repairs to CCTV, tele- 
phone systems, and standard recording equipment) are relatively low, 
although the services of skilled technicians may be frequently required. 
Aggregate costs will be far from negligible; it is only in comparison 
with other laboratory types that they are low. 

Measurement potential: The measurement potential of the manual 

facility is, typically, both wide in range and crude in precision. It 
is wide in the sense that almost all types of data can be collected — 
practically nothing is excluded by reason of instrumentation; or, 
conversely, some off-the-shelf instrumentation can be found for almost 
any data of interest. The resultant data tend, however, to be either 
unwieldy chronologies that require extensive data reduction, or 
focused measures that are too gross for fine-grain analysis. The 
inability to achieve the levels of measurement demanded by many lines 
of inquiry is clearly a disadvantage of the manual laboratory. 

Adaptability: This type of facility is very adaptable and flexible, 

in that different kinds of research can be implemented without the need 
for extensive technical suppoir. . The adaptation of a research design 
into the laboratory setting can be comparatively rapid and easy, 



0 



30 



requiring only different sets of displays, different rules of operation, 
and different space and equipment configurations. Since displays and 
rule formulations do not have to be reflected into hardware or soft- 
ware, they can be produced or modified without great difficulty. 
Equipment, because it is standardized, presents few interface problems, 
thus lending itself to rapid reconfiguration. 

2. Special Purpose Laboratories . Since the manual laboratory is 
confined, by definition, to off-the-shelf devices, its level of refine- 
ment is limited. Greater precision in investigation can be achieved 
by introducing special purpose equipment, and then the upper bound on 
refinement and sophistication will be essentially the practical one 
of capital investment. As the name "special purpose" implies, the 
equipment tends to be tailormade and project-specific. The limiting 
factor is that the utility of a given line of investigation must 
justify the expenditures necessary for it. Generally the special 
purpose laboratory is engineered and project-specific, with significant 
portions of control and operation committed to mechanical devices. 

Costs: The initial investment for this type of facility varies greatly 

with the research purpose, but a general observation can be made about 
amortization: no matter whether it is a complicated mock-up of a 

complex system or special equipment for a simple two-choice experiment, 
most of the capital costs must be amortized in the life of a single 
project. Operational and maintenance costs will also vary with the 
complexity of the equipment; but, again generally, the specialized 
nature of the equipment will tend to require more technical support 
than standardized off-the-shelf devices. Production is largely mechani- 
cal, so production costs can be relatively small. 

Measurement potential: The special purpose laboratory, as a departure 

from the manual type, is directly and primarily addressed to the problem 
of achieving more precise situations and means for their measurement; 
potential in this direction is, then, almost directly related to 
resources available for development. It is therefore difficult to make 
any but a relative assessment of measurement potential — almost any 
degree or type of measurement is possible if the costs can be justified. 
This principle clearly has a logical upper bound when the desired degree 
or type of measurement is beyond the technological and/or methodological 
state of the art. A practical upper bound is encountered when the 
measurement requirements can only be satisfied with equipment having the 
data processing capacities of a large scale computer; at this point 
the function of the laboratory is usually, though not always, reconceived 
as a general purpose, computer-based laboratory. 

Adaptability: The greatest liability of a special purpose laboratory 

is, patently, its singleness of purpose, usually not just restricted 
to an area of study but to a specific line of inquiry. Adaptation to 
other investigations generally requires significant redesign and re- 
development effort. 
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1. Computer-based Labora tory: Dedica ted Computer. The dedicated 

computer-based laboratory uses a digital computer, with associated general 
purpose input and output devices, to support research. The computer can 
sequence events to be presented, generate information to be displayed, 
format and display this information (usually on cathode ray tube display 
devices), accept responses from subjects, and record sequences of events 

and responses. 



These general capacities are adapted to specific research needs through 
computer programming (software) . The capacity of the computing equip- 
ment may be limiting, but more often the source of limitation resides 
with software. Generally the computer based laboratory is a complex 
of general purpose equipment which is capable of displaying and gather- 
ing information, but these capabilities must be applied to specific 
needs through extensive computer programming. The usual aim is for 
control and operation to be entirely automated, that is, to become a 

computer function. 

Costs* The initial investment costs of the dedicated computer laboratory 
are relatively high. In addition to acquisition of the computation system 
(the computer and its complement of peripheral equipment— general ^ 

storage devices, general input and output mechanisms), the a oratory 
components must be incorporated into the complex. Participant display 
and response devices usually require design and fabrication of special 
computer-to-subject interface equipment and a general laboratory operat- 
ing system of computer programs must be developed. In all, the 
initial expenditures are high, but due to the general purpose nature 
of the system they can be amortized over all laboratory users. Pro- 
duction costs for a given study will involve additional computer pro- 
gramming. Variations within a line of inquiry can be handled by simple 
parameter or control settings, but the programming required for a 
newly formulated study will require additional man-months of work by 
experienced programmers and significant amounts of developments 
computer time. Maintenance costs involve a computer maintenance 
contract, laboratory technicians, and system programming staff. Again, 

these costs are relatively high but can be amortize over ® t up S hundreds 
(derating costs— primarily the cost of the computer— run in the hundreds 

of dollars per hour. Such costs would be prohibitive °" 

for substantial numbers of participants in a given run; w , 

station laboratory the participant-hour costs begin to approach those 
of many special purpose and manual facilities. 

Measurement potential: The measurement potential of a computer-based 

laboratory is generally unexcelled. Since the centra pr°c 8 
unit is primarily designed for digital and symbolic manipulation 
it can handle a wide range of data with great precision. The fact that 
La must be digitalized is not a significant limitation, since even 
though certain situations may require analogue devices tor greatest 
simplicity and precision, digitalization is always possible. It is, 
however, in terL of range that the computer-based facility presents 
the greatest advantage. Of particular significance for behavioral 
investigation is the ability to handle language in written form--the 
ability to store, retrieve, and present texts-and, to a limited 
extent, to interact linguistically with participants. 
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Adaptability: Caution must be exercised in assessing the adaptability 

of a computer-based laboratory. Many factors seem to point to extreme 
flexibility and generality: the computer itself is general purpose 

in nature and the participant presentation and response devices are 
also usually general purpose, so that only the software need change 
from one study to the next. This characterizrtion is accurate in the 
sense that two entirely different studies (say, an economic trade game 
and a programmed instruction study) can be run back-to-back with only 
the software changing (a process that is measured in minutes); but 
this is deceiving in the sense that one completely developed investiga- 
tion program is being swapped for another. Adaptability must also be 
assessed in terms of development — the delay between the time a project 
is designed on paper and the time it becomes implemented in the 
laboratory. The period required for computer programming formulation, 
coding, debugging and check-out can be a deterrent to bringing studies 
into the laboratory. In this regard a computer-based laboratory shows 
no particular advantage over the special purpose laboratory. 

4. Computer-based Laboratory: Time-shared Computer . The time- 

shared computer-based laboratory performs the functions described above 
almost as if it had a computer dedicated to it alone. However, it is 
only one user among many simultaneous users, thus sharing the opera- 
tions, capacities and costs of a large-scale computer. 

Costs: The initial, maintenance, and production costs for a time- 

sharing computer are comparable to those of a dedicated computer 
facility. Any difference would be in the direction of increased costs 
since time-sharing systems are more complex than dedicated types. It 
is, however, in the area of operating costs that a substantial saving 
is realized, since these costs are shared by all concurrent users of 
the computer. Overall, this economy is sufficient to make time- 
shared computer-based laboratory facilities generally competitive with 
special purpose and manual laboratories. 

Measurement potential: Again the measurement considerations are rough- 

ly equivalent to those already discussed for a dedicated computer lab- 
oratory. With regard to timing precision there is some loss due to 
the fact that a time-sharing user does not have exclusive access to 
the control of the machine; therefore there can be no guarantee that 
the program will return to given user within a specified period of time. 
Studies with narrow tolerance in timing ra^y be pre-empted from a time- 
shared laboratory. 

Adaptability: Adaptability considerations are essentially the same as 

with a dedicated computer, since time-sharing characteristics do not 
as yet usually significantly facilitate interstudy turnaround or intra- 
study development effort. However, time-sharing has begun to make 
on-line, interactive, development increasingly possible. Eventually 
this could lead to a considerable increase in adaptability. 
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E. Computer-Based Laboratory Capabilities 



The power and promise of the computer-based laboratory can be reviewed 
under the following five headings, which represent general dimensions 
of laboratory investigation: stimulus generation, response monitoring, 

response network, respondent role, and presentation/response range. 
These five aspects can be further subdivided as indicated below. 

- Stimulus generation : definition of presentation content by program 

(system, experimental vehicle, instructional 
vehicle) . 



Static: presentation content prescribed; generally not 

modified by participant interaction with program. 

Dynamic: presentation content determined in degree by 

participant interaction with program. 

- Response monitoring : frequency with which participant (user, subject, 

student) responses need to be monitored by the 
program (system, experimental vehicle, instruc- 
tion vehicle) . 



Periodic: monitoring only at specified occasions. 

Sporadic: occasional but irregular monitoring — participant 

and/or program instigated. 



Continuous: uninterrupted monitoring. 



- Response network : relationship between response sets of different 

participants . 



Single or parallelism in response sets; all participants 

Unison: always at same nexus in program. 

Temporally all participants traverse substantially the 
Independent: same paths in the program, but rates are individ- 

ually determined. 



Content participants can traverse individually different 

Independent: paths in program. 

Interactive: participants (at least two) are potentially 

interdependent in evaluation of their response 
sets; group-produced phenomena are a special 
subclass of the interactive response network. 

- Respondent role: relationship between the program (system,^ 

experimental vehicle, instruction vehicle) and 
the participant (user, subject, student) in 
terms of the participants' conscious control 
over program delineation. 
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Reactive: participant reacts to a prescribed program. 

Directive: participant directs (at least in part) program 

development . 



- Presentation / 
response range : 



modes and type of presentation (situation, 
problem, stimulus) to participant and response 
(action, answer, response) from him. 



Narrow: 



presentation and/or response prescribed and 
limited. 



Wide: presentation and/or response open-ended, 

essentially controlled by participant. 

Special: presentation and/or response not necessarily 

prescribed but circumscribed, usually by 
participant abilities or research content. 

In order to anchor this framework with a concrete example, one of the 
better known multi-media approaches to education, computer-aided 
instruction (CAI), will be characterized in terms of the foregoing 
distinctions. The initial attraction to CAI was clearly based on the 
capacity for producing tutor-like characteristics of continuous 
response monitoring within a temporally independent response network. 

The deficiencies of the earliest and most rudimentary forms of CAI 
were, as clearly, in the areas of respondent role (the almost totally 
reactive aspects were considered limiting) and of presentation/response 
range (they were typically restricted to static, usually discursive, 
presentations and multichoice button responses) . Developmental efforts 
in CAI since have largely been directed toward extending the response 
network to include content independence (branching and multi-subject 
content vehicles) , toward making the respondent role more directive 
(permitting student definition of problem areas, providing assistance 
to the student at his initiative, etc.), and toward widening the 
presentation/response range (utilizing many audio-visual devices for 
presentation and incorporating — some with tolerance for insignificant 
error, ambiguity and synonymity — simply composed responses). It might 
also be noted that considerable work has been done with specialized 
presentation/response ranges especially as they are related to participant 
abilities; non-literate students have been accommodated with auditory 
and graphic presentations and with pointing and positioning responses 
through lignt pen and joy-stick devices. Finally, there has been 
some limited development in CAI of interactive response networks. 
Computerized versions of group problem solving tasks, simulations, 
and other pedagogical "games" would fall in this class. 

While this characterization of CAI efforts has been primarily intended 
as .1 grounding example for the investigatory classification scheme, 
even such a cursory review is telling with regard to the differential 
contributions of hardware and software in determining research capabili- 
ties. With computerization as a given, an instruction program achieves 
continuous monitoring and (at least) a temporally independent response 
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network nearly straightway, but further sophistications depend almost 
exclusively on software development. A possible exception appears to 
be related to the improvements in the present ation/response range; 
and while these may involve the incorporation of new hardware devices, 
software modifications are usually required for implementation and are 
always necessary for full utilization. In short, the definitive aspects 
of research capability in a computer context are largely related to 
software. 

It should be noted that the foregoing discussion of CAI illustrates the 
computer in a dual role, namely both as an educational medium and as 
a laboratory tool. However, the illustrated dimensions of investigation 
are equally present in situations in which computers have only ins f u- 
mental (not instructional) roles, as well as in manual laboratories 
where no computer is present. The following paragraphs summarize some 
of the more generic characteristics of the various types of simulation 
laboratory facilities, as related to the different dimensions of 
research. 

Stimulus generation. Manual and special purpose laboratories can be 
adequate for the static type of stimulus generation. For the dynamic 
type of stimulus, however, manual and special purpose laboratories are 
severely limited; extensive pre-planning and considerable ingenuity 
are required for even a comparatively modest achievement. Computer- 
based laboratories have virtually unlimited stimulus generation 
capabilities, provided that enough money is available for necessary 
development effort. 

Response monitoring . A manual laboratory is optimally cost-effective 
for periodic response monitoring; a time-shared computer for sporadic 
monitoring; a dedicated computer for continuous monitoring. A manual 
laboratory requires a high experimenter-to-participant ratio for 
continuous monitoring, and then encounters significant data-reduction 
problems. A special purpose laboratory can overcome many of these with 
sufficient investment in special equipment. Time-sharing makes continuous 
monitoring by computer impossible, but often an adequate approximation 
to continuity can be devised. 

Response network . For a single or unison response mode, the manual and 
the special purpose laboratory can be adequate. For more complex net- 
works, the manual laboratory will generally soon lose control, and 
more or less extensive special purpose equipment investments will be 
necessary. Computer-based laboratories can handle all types of response 
networks, although elaborate software development is likely to be 
required, especially in order to achieve the content-independent and 
the interaction capabilities. 

Respondent role . For the directive mode, in which the participant 
partially directs program development, the manual laboratory is current- 
ly the most cost-effective, eliminating needs for extensive equipment 
or software development required by the other laboratories. For the mode 
in which the participant reacts to a prescribed program, the manual 
laboratory is generally the least satisfactory of the four types of 
laboratories . 



o 

ERIC 



36 



Presentation/response range . For the narrow range, standard equipment 
and/or software is sufficient. For wide and special ranges, either the 
flexibility of the manual laboratory is required, or specialized 
equipment (special purpose laboratory) or specialized software adapta- 
tion of general purpose equipment (computer-based laboratory) must be 
provided. 

From the foregoing paragraphs it is obvious that on the whole a time- 
shared, computer-based laboratory is the optimal single facility for 
the types of investigation being considered. For direct support of 
research studies its capabilities are nearly equivalent to those of 
a dedicated computerized laboratory, the major difference being with 
respect to the periodicity of stimulus generation and response monitor- 
ing. From the point of view of operational economy, the time-shared 
computer-based laboratory is clearly superior. 

Beyond this general assessment the major comparison to be made between 
laboratory capabilities relates to their potentials for extending 
research in new directions. Such estimations are tenuous because they 
do not, and they cannot, take account of the ingenuity of the investiga^ 
tor. A clever and inventive researcher in the most rudimentary manual 
laboratory would probably be more productive than a plodding hack 
surrounded by the most sophisticated electronics. This critical un- 
known aside, it is possible to draw some very general conclusions about 
research potentials strictly from the characteristics of the physical 
facilities. With this provision one can say that certain research 
possibilities are virtually excluded by the limitations of a non- 
computerized laboratory. 

A seemingly important class of studies in terms of educational import 
are those in which social interaction phenomena are central. To the 
extent that social skills and interpersonal behaviors intercept with 
the educational process, interaction studies are a viable area of 
investigation. Control and strict accounting of interpersonal inter- 
actions almost requires the power of computer control. Thus, if 
manipulation is an essential aspect of inquiry — a need, say, to insure 
that peripheral members would assume a position of centrality in a 
group task — this presupposes a degree of control that is best realized 
through computer mediation capabilities. By and large, interactive 
studies have been restricted to measurement (e.g. , sociograms) or 
gross manipulation (e.g., Sherif and Lewin) because on-line controls 
and assessments are nearly impossible without computer assistance. 
Positive behavior modification in the interpersonal area generally 
requires closer control and observation than have heretofore been 
available; in this respect the computerized laboratory could open up 
an important new area both to training and research. 

Comparable observations can be made with regard to investigations that 
have situational complexity as an essential of study design. One can 
conceive studies or instructional procedures, for instance, that are 
specifically designed to aid the participant by minimizing his own 
need to have otherwise requisite skills for a general task — a design, 
say, to exercise decision-making with minima!) accompanying demands on 
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immediate memory, or problem solving with little or no need for computa- 
tion. In either case the technique for minimization might be to supply 
the participant with aids for those prerequisite but nonessential 
aspects of the general task. It is easy to see how this might be 
accomplished by putting electronic data processing capabilities at the 
respondent’s command (the computer could handle retrieval cc computa- 
tion on request). In general, for complex simulations the advantages 
of computer administration are immediately obvious — data representing 
the environment can be maintained, updated and retrieved automatically 
or on demand; observational daca gathering and recording can be placed 
under program control, and generally the computer can assume any of 
those functions that the investigators find to be obstructive to the 
purposes of their study. Once these obstacles are overcome new 
possibilities emerge. New sources of data can be tapped (e.g. continuous 
interviewing controlled and administered by the computer) , new uses of 
simulation can be explored (e.g., a simulation history and a real 
history with the same origin can be run in parallel for pedagogical 
purposes), new techniques can be used (e.g., extrapolation and inter- 
polation guidance can be offered to the participants). The possibili- 
ties are practically as boundless as the investigator’s imagination, 
but they will not emerge as long as the investigator is nearly taxed 
to capacity with the mechanics of the simulation. 

A computer-based laboratory presents another distinct advantage over 
noncomputerized facilities in that just by virtue of the computer’s 
presence, it is possible to have adjunct support for evaluation. 

Physical presence bears relevantly on the case of coordination 
between the investigator and those developing and maintaining the data 
management programs. The closer the relationship between the researcher 
and the data analysis services, the greater the potential for developing 
tools that are of real use: productivity can be increased simply by 
housing both functions in the same facility. 

Computer-based simulations offer a new approach to research which can 
be called "structured induction". The structured induction approach 
combines the merits of induction with the analytic merits of formal 
theoretical guidance; that is, it allows the researcher to recreate a 
practical concrete setting in the laboratory, and within these embedd- 
ing conditions, to design decision situations that conform to a formally 
described decision model. This can be seen, for example, in a compari- 
son of the evaluative tools required for data handling under the 
"classical" and the "structured" inductive approach. 

In raw form the results of the classical inductive method are complete 
data records; the aim, of course, is to find descriptive generaliza- 
tions that will account for the recorded particulars. At the simpxest 
level the procedures are obvious — a straightforward test of whether 
all instances can be unexcept ionally accounted for by a summary 
generalization; but positive results are rarely obtained at the simplest 
level. Beyond that there are two eff< ctive but difficult procedures 
for finding generalizations. One procedure simplifies the data by 
reconceptualizing them, so that otherwise diverse data are counted as 
comparable by virtue of an underlying commonality; the second procedure 
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is to introduce complexity and contingency into the generalization, 
so that otherwise exceptional instances can be incorporated on the basis 
of explicit qualification, These are difficult procedures to support. 
Reconceptualizing data, even supposing the requisite formation of 
concept, entails the need to transform all pertinent instances. This 
type of data manipulation has not been generally developed; for the most 
part it has been left to the energies of the investigator and his 
assistants. The second procedure of introducing complexity into the 
generalization — which is entirely compatible with the procedure of 
simplifying reconception—presents sizeable practical difficulties 
without mechanized assistance. When the generalization reaches even 
modest degrees of complexity, it becomes virtually impossible to 
evaluate by hand. The only practical recourse then is to forego the 
inductive aims or ignore the exceptional cases. 

In order to support the inductive method there is clearly a need to 
develop better evaluative tools. This is possible through a program of 
data management within the framework of structured, induction. There 
is a need for data analysis support beyond that which is provided by 
standard statistical evaluation. A basic statistical capability would 
be required in any event, but that can be obtained, with computer 
capability, almost for the asking. The additional need is to develop 
new evaluative tools especially to meet the needs of inductive 
analysis. At a minimum there is need to attend to two separate but 
complementary aspects of the problem: (1) to develop computer programs 

for the management and manipulation of data, (2) to develop means of 
computer assistance in the formulation phase of inductive analysis. 

The first would afford the researcher greater facility in handling 
the masses of data that he encounters; the second would provide him 
with inductive algorithms to aid him in finding relationships in his 
data. Some preliminary steps in both of these directions have already 
been undertaken; a brief description of each is given to underscore 
the possible lines of development. 

Computer data management and manipulation . Data are being amassed 
in unprecedented amounts in both the public and private sectors of 
our society. This fact is related in an important way to the advent 
of electronic data processing. With an expanding society there would 
be simply a proportional rise in data growth, but with advanced data 
acquisition techniques there has been a significant increase in the 
amount of potentially accessible data. 

The potential value of this vast increase of information is great— 
but too often this potential goes unused because the overwhelming 
abundance of data makes even the least complex approaches to organiza- 
tion, collation and assessment exceeding expensive and time-consuming. 



Conventional approaches are of little assistance in inducing or dis- 
covering a fundamental order in such a body of data. Techniques such 
as factor analysis, for example, are only marginally effective, especial- 
ly where assessment calls for dividing the data in different ways, 
omitting selected subclasses from comparisons, using different observed 
values in operational definitions, and repeatedly recombining, regrouping 
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and recalculating to observe the effects of manipulation. 

TRACE — Time-Shared Routines for Analysis, Classification, and Evalua- 
tion — is a computer-based system for assessing and structuring large, 
complex sets of data prior to subjecting them to statistical processing. 
TRACE was created in an attempt to solve these data analysis problems. 

The system assists the investigator by enabling him to explore, from 
a number of different or newly suggested points of view, the relation- 
ships that may prevail among complex sets of data. The investigator 
is in direct and essentially continuous contact with the computer via 
teletype and cathode ray tube display. He receives immediate feedback 
of the results of each computation or manipulation and so may modify 
his analyses as he proceeds. The end effect is an efficient interplay 
between the investigator's conjectural and judgmental skills and the 
computer' c capacity for rapid and accurate data processing. 

Computer-aided induction . Classical induction algorithms such as factor 
analysis, multiple regression, and early forms of cluster analysis 
have been employed successfully to reveal a limited class of structures 
for particular classes of data. Even contemporary forms of cluster 
analysis, as used for example in numerical taxonomy, are restricted 
either in the structure to which they can respond or in the type of 
data to which they apply, or in both respects. However, when the 
structure or pattern becomes less well defined and less well specified, 
man's experience, hunches, and intuitions often enable him to function 
where such algorithms fail to apply. In order to overcome the limita- 
tions and restrictions of pure machine-induction, we are seeking a 
workable interplay between the investigator's judgment about and know- 
ledge of his d~ta, and the dal. Uc mdling power of a time-shared computer. 
To this end a computer-induction program, IDEA, has been designed 
either to run on its own, or to interact with the investigator by 
(1) presenting him with the facts he requires to evaluate each major 
"decision" in the analysis, and then (2) accepting his concurrence or 
overriding judgment before continuing on to the next major decision. 

In addition to its "interactive mode", IDEA has two other distinguish- 
ing features: (a) heuristic computation procedures are used in those 

cases where the combinatorial aspects of the analysis would require 
extensive computation, and (b) different heuristics are used for 
different types of data, enabling IDEA to operate on a mixture of 
nominal (categorical), ordinal (ranked), and interval or ratio-scaled 
measurene its . 

In summary , the IDEA concept is concerned with the development of 
an interactive package of heuristic computer-induction programs. The 
resulting system should enable an investigator tc explore his data base 
in search cf any structure inherent in his data. In particular, the 
search is for those relations among his variables that best enable him 
to nreviict any variable of interest from a knowledge of the data on the 
other variables. It appears that such interplay between the investigator 
and his data holds promise for a more effective inductive analysis 
than either man or algorithm could produce a Lone. 
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F. Feasibility of a Computer-Based Multi-Media Simulation Laboratory 



The potentialities of a simulation laboratory (in particular, one 
supported by a large-scale digital computer) for approaches to multi- 
media educational problems have been described above under several 
separate headings. These explorations can be brought together here 
by proposing a maximally-ef fective configuration for a laboratory, by 
presenting examples of projects which could be undertaken within this 
configuration, and finally by assessing such a laboratory's utility. 

A semi-manual, computer-based laboratory is here proposed as a maximally - 
effective configuration for the types of laboratory functions hereto- 
fore discussed. From the foregoing discussions of types of simulation 
laboratory facilities, the most apparent conclusion to be drawn is that 
the greatest assets lie on either end of the continuum. On the one 
hand the manual laboratory is the most adaptable, with particular 
strength in low production costs and short implementation times; on 
the other hand the time-shared computer-based laboratory provides 
a wide range of measurement potential, sufficient degree of precision 
for all but a limited number of studies, and great efficiency in 
operating costs. The most promising laboratory facilities, then, would 
combine the best features of both. 

The utility of a semi-manual, computer-based laboratory can not be fully 
understood apart from the process of developing a new line of investi- 
gation. Ideally, research can approach experimentation through successive 
stages of investigation — an idea, a paper-and-pencil version, perhaps 
a breadboard model, and finally full instrumentation — with each step 
affording an opportunity to refine the formulation and procedures. 

For simpler studies this approach works well; but for more complex 
studies, especially where the data handling capabilities of a computer 
are required even in part, the process tends to be foreshortened — the 
intermediate steps between initial broad conception and final implementa- 
tion drop out. This occurs because the natural but unnecessary tendency 
is to conceive of the study as wholly computerized if computerized 
at all, and this conception carries an implicit commitment to a nearly 
complete programming formulation and specification of the research 
vehicle at the outset. The investigator must arm-chair the investiga- 
tion without benefit of the concretized intermediate stages of develop- 
ment; but since most investigators are not well conversant with computer 
capabilities, this task is made all the more difficult. There is, in 
short, a frustrating problem of interface between the investigator and 
the laboratory programming. This problem is further complicated because 
preconception of a wholly computerized mode of operation entails a 
commitment to the limitations of electronic data processing, and some of 
these, ironically, present little or no problem for human judgment. 

This point can be generally illustrated with any study permitting 
freely composed responses from the participant— say , an instruction 
program employing recall test modes. It is extremely difficult to 
program around the problems of recognizing correct but not exact 
answers (misspelling, synonymity, ambiguity, etc.), since computers 
evaluate by absolute matching. The problem is not insurmountable, 
but is an inordinately difficult obstacle in the developmental phase 
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of the investigation since the matter of inexactness would presumably 
be only incidental to the main line of inquiry. This problem can be 
temporarily if not permanently circumvented by simply looping the re- 
sponse through a nexus of human judgment in the system; but this 
expedient, simple in concept and execution, is usually bypassed 
because of the overriding commitment to complete computer implementation. 
This problem is not isolated or unique. Since electronic data process- 
ing has no tolerance for the sorts of inexactness that humans find 
not only tolerable but highly functional and, within limits, adequate 
to their purposes, investigators are often chagrined to find that 
direct and complete computer handling of even modest studies would 
incidentally entail full understanding of fundamental cognitive 
processes. This, of course, leads to compromise, distraction, or 
withdrawal. In any event the computer that is optimal for certain 
aspects of the investigation becomes unduly obstructive in other 
respects . 

The reconception of the computer-based laboratory as a manual type of 
facility with special information handling characteristics permits the 
user to redefine the process of investigation development from an 
armchair procedure to one of successive approximation toward the final 
product — looking for the best mix of computer and human processing and 
controls. In effect, this procedure allows the researcher, by 
essentially simulating his investigation, to interact directly with 
the laboratory vehicle almost from the outset of its development. 

All of this presupposes a general program that will permit quick entry 
into the laboratory; such a program would represent an all-channel, 
computer-mediated network with one or more of the positions assigned 
to experimenters. Communication paths could be closed by parameter 
settings to produce the operational network desired; initially all 
but information presentation and data gathering functions would be 
manually controlled; those functions that prove to be too difficult 
for humans to fulfill can be assigned to the computer until, in the 
judgment of the investigator, an optimal mix has been achieved. 

This developmental procedure would be applicable to all types and 
levels of investigation but its greatest utility would be realized with 
studies of moderate to great complexity (simulations, complex games 
or multi-condition experiments) where, without the intermediate steps 
of development and freedom from commitment to an all-computer vehicle, 
research aims tend to become obscured in the process of implementation. 

The activities of a laboratory constructed along the foregoing lines 
can be illustrated by hypothetical examples of studies for which the 
laboratory would be appropriate. The hypothetical studies described 
below are drawn from five sample problem areas ; education of the 
handicapped (blind students), education in the sciences (fostering 
inquiry skills), education of the culturally disadvantaged, means of 
extending teacher effectiveness, and communications/decision-making 
in international relations. Each example is designed to illustrate 
a more general point, namely, that extended research facilities will 
open up new techniques of investigation and training — not necessarily 
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in the specific ways embodied by these example studies but, as these 
studies illustrate, by enabling the researcher to range beyond current 
methods. Generally, then, these examples can all be characterized 
as investigations requiring a degree of control and manipulation of 
complex and/or social processes that would make optimum use, in a 
self-contained laboratory setting, of the semi-manual computer-based 
facilities described above. 

1. Education of the Handicapped . The blind student represents 

an especially difficult problem for educational institutions because his 
most pronounced and debilitating deficiency is in the area of social 
skills. In this respect, the blind student and the deaf student show 
quite different patterns of development. The deaf are often mistake ly 
characterized as mentally deficient since auditory experiences appear 
to be especially critical to early stages of intellectual development. 
Blind children, on the other hand show (with the proper adjustments 
in methods of assessment, of course) the normal range of intellectual 
skills. Given the proper techniques for overcoming the obvious handi- 
cap in reading, schools have been able to effect a near parity in 
intellectual opportunity for the blind student; this, coupled with 
more intensified attention and a compensating freedom from visual 
distractors, has sometimes produced even better than normal intellectual 
development. The problems for the blind are not primarily in the 
intellectual areas but rather in the social. Once they overcome the 
problem of reading, they are intellectually restricted only by the 
availability of Braille materials; but there is not comparable assist- 
ance in areas of social interaction. Ironically they can more easily 
learn to relate to books than to people. 

In at least two respects a computer-based laboratory might prove 
effective in dealing with this situation. First, in a computer-based 
laboratory with isolated participant stations, it is possible to 
create social interaction situations that have no face-to-face relation- 
ships. Whereas investigators normally employ such situations as control 
devices, it is apparent that these conditions are the blind child’s 
standard social milieu. It is thus speculated that one could create 
a "natural" social context for the non-sighted person which would seem 
to be basic in any attempt to develop something like Braille techniques 
applicable to social skills. Secondly, the problem of communication 
for the blind in such a setting is well within the capabilities of the 
computer, since translation to paper tape output in Braille form (as 
opposed to visually reproduced material) has already been accomplished — 
computer generated Braille is already within the current state-of-the- 
art. Thus, a computer based laboratory presents possibilities for 
social skill training of the blind that are particularly appropriate — 
in terms of context, control and capabilities — to one of the most 
crucial problems in education of those with visual handicaps. 

2. Education in the Sciences . Science education, particularly 
at the secondary education level, has been greatly augmented and 
refined under the encouragement provided by NDEA programs. One of 
the major thrusts in this development has been directed toward the 
processes of inquiry in contrast to conclusions, toward method in 
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contrast to content. This is a particularly difficult area for 
instruction, not only because we know less about the process of 
discovery than we know about discoveries that have already been made, 
but also the because instruction task itself, to be effective, must 
be dynamic, so that it is difficult to manage on even a moderately 
large scale. By definition, inquiry training requires a responsive 
environment; but by using nature as the environment, the instructor 
relinquishes much in terms of control and assessment in the situation. 
Especially critical is the fact that the student, already familiar 
with the environment, may through his prior knowledge, pre-empt the 
inquiry learning experience. 

All of these considerations seem to point in one direction of simula- 
tion — toward a situation involving a responsive environment that is 
content-free and yet under the instructor’s immediate surveillance and 
control. This type of environment can be created in a computer-based 
laboratory. A different world can be represented internally in compu- 
te:; programs; it can be responsive to trial and error, to hypothesis 
testing, and even, where appropriate, to theory building; overt 
inquiries can all be recorded for subsequent assessment and analysis, 
and generalized aids to inquiry (guidance procedures) could be offer- 
ed if desired. Along with this versatility it would be a large scale 
teaching device. The creation of such an inquiry vehicle would be 
a difficult undertaking; but even if one had such a model in hand, 
its dynamic, responsive and complex nature would preclude its being 
managed in any setting other than one that is computer supported. 

3. Education of the Culturally Disadvantaged . The special educa- 
tional needs of the culturally disadvantaged child are matters of 
increasing concern for the school system. There is a growing awareness 
that remediation — even the best sort of remediation in the usual sense 
is not sufficient, because it presupposes more rudimentary skills that 
have heretofore been taken for granted. Thus, many reading difficulties 
are now seen as being antedated by lack of exposure to the most basic 
communication experiences (e.g. not having heard many of the words 
that comprise the listening and/or speaking vocabulary of most middle 
class children). In many regards, then, the educational programs for 
the culturally disadvantaged can be characterized as explicitly adapting 
to the absence of experiences that were implicitly assumed by standard 
educational practices. Where these experiences are simple, the means 
of basic remediation can be direct — either basic communication experi- 
ences can be provided by direct exposure to something not unlike the 
everyday environment of a middle class home, or reading instruction 
can be provided in the children’s own language (instead of in the 
foreign middle class language) . But where the experiences are more 
complex (e.g. deferred reward experiences, social efficacy experiences, 
etc.) the same sort of direct approach does not seem practical. 

Where the direct approach is not practical, the most promising indirect, 
but still concrete and personalized, experience would seem to lie in 
simulation. If it is not practical to make the disadvantaged child's 
father a status figure in the community for the child’s educational 
benefit, it is nonetheless possible to have the child himself seriously 
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play the social and economic "games" of the dominant culture. It would 
be possible to give the child employment experiences while he still 
has room educationally to benefit from the preview; it would be possible 
to reverse the roles (so that he would be interviewing himself) to 
better appreciate the employer’s concerns and requirements; generally, 
in fact, it would be possible to give him a range of posteducational 
experiences to which he could relate his educational progress. But 
in all this it would be especially important to be able to manipulate 
and closely control the simulation in order to preclude a reversion 
to more familiar and comfortable modes of response; in other words, 
a "free-floating" simulation game would not be as effective as one in 
which the behavior was bounded and directed toward specific objectives. 

The degree of manipulation required for this sort of "structured" 
experience would require a type of logical control available only with 
very sophisticated laboratory resources. 

4. Means of Extending Teacher Effectiveness . The need to extend 
teacher effectiveness stems from two sources. One is circumstantial — 
the shortage of qualified personnel and their cost of employment 
means that the available teacher's efforts will be dispersed among 
more students; the other is categorical — quality of instruction is 
always open to improvement under any circumstance, especially in light 
of constantly improving technology and understanding of the learning 
process. Those who approach the teacher effectiveness problem from 

a research perspective are generally prescriptive in their orientation — 
the results of design, testing and evaluation are passed on as implicit 
or explicit recommendations for practice. In the multi-media context 
this means testing the relative effectiveness of new methods using 
recent advances in technology, and then handing them over to the 
educational system as a package for implementation. (The implicit 
assumption of this orientation is that the demonstrated superiority 
of the multi-media context will be the primary motivating factor for 
the system's adopting a multi-media approach). A contrasting orienta- 
tion to the problem is the "enabling" approach, where the intent is 
to make the technology itself directly available to the practitioner's 
use. This contrast is evidenced in the most recent developments in 
the audio-visual field; many institutions now provide instructional 
aid centers where the emphasis is on constructing one's own aids 
rather than selecting from a library of pre-packaged aids. An 
analogous emphasis with regard to computer supported instruction 
would have implications for an approach to the problem of extending 
teacher effectiveness. 

Looking at the computer-based laboratory as a prototype of future class- 
room facilities, one might explore in a direct manner the question of 
how to enable practitioners to exploit this potential to their own designs. 
The laboratory in this type of investigation would become the major 
independent variable in a meta-research design and hence an indispensable 
factor in this approach to extending the effectiveness of instruction. 

5 . Communications /Decision-Making in International Relationships . 
Research into decision-making serves well to illustrate the structured 
induction approach within the context of this project. This is true 




45 



both because decision-making is a critical process which pervades the 
educational experiences of students, teachers, and administrators, 
and because the study of decision-making illustrates many of the 
subtleties and complexities appropriate to laboratory simulation. 

Classical theories have required the decision-maker to have complete 
and accurate knowledge about the consequences of all decision alter- 
natives — a distinctly unrealistic assumption. Some recent decision 
theories formally take uncertainty into account and relax several 
other assumptions, but their mathematical rigor still requires rather 
stringent assumptions about the decision-maker; for example, that the 
decision maker 1) never overlooks a message, 2) extracts from a message 
all the information it bears, 3) names things properly and at the 
appropriate times, 4) forgets nothing, 5) stores and recalls information 
without distorting it, and 6) never acts on preset rules but always 
on the basis of the assessed consequences of a choice which best 
realizes the chances of achieving his goals. Such assumptions are 
also clearly unrealistic when applied to mortal man in practical 
situations. Thus, none of the existing mathematical theories seems 
to offer a suitable model of communication-decision situations which 
could be applied to the real world. 

Most experimental studies of decision-making also suffer from over- 
simplification, for the sake of experimental rigor, to the point where 
they too have little connection with real life decision-phenomena. 
Furthermore, the traditional inductive approach, based on memories 
of actors in the situation, on naturalistic observations, on historical 
case studies, and so forth, is also of limited scientific value. 

Simulation, at its best, offers the researcher a chance to admit selected 
elements of reality into the laboratory, where they may be controlled 
and measured without the time-consuming frustrations produced by the 
unpredictable variability of the real world outside. In short, the 
simulation laboratory allows the researcher to combine merits of 
induction with the analytic merits of formal theoretical guidance. 

It allows the researcher to recreate a practical concrete setting in 
the laboratory, and within these embedding conditions, to design 
decision situations that conform to a formally described decision model. 

There is some modern theory which can help, such as the formulations of 
March and fimon. Here the concept of "rational man" is shifted to 
that of "limited rational man", who attempts to find not the best 
alternative, but simply one which is "good enough" for his purposes. 

A distinction is made between decisions made by rule-following and 
decision situations proper — where the evaluations of alternatives are 
many, complex, and not easily summarized; where relative merits seem- 
ingly shift unpredictablv with unique configurations of the environment; 
or where no available alternative clearly and uniquely leads to desired 
goals. The latter cannot be delineated by specific rules, since even 
if rule-following routines could be enumerated, books the size of 
telephone directories vould be needed to list the rules for all conceivable 
combinations of circumstances. 
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Another shift toward realism is reflected in viewing the communication 
decision process as the full span of activity, from detection of the 
decision situation, through the act of choosing, and post-decision 
behavior. This shift in emphasis brings into focus the entire process 
of search and evaluation which leads to choice. 

With the use of a computer-based laboratory, each decision situation, 
the subjects’ perception of the situation, the consequences noted, the 
alternatives omitted and considered, the evaluations made of these, his 
search, and finally his choices can be coded. These coded perceptions 
and interactions can then be time-ordered, and matched or contrasted 
with real life situations or with designed situations. Basic predic- 
tions derived from theory attempting to explain these situations can be 
checked, and new hypotheses can be generated. 

This approach cannot be celled rigorous hypothesis testing at this 
stage, since the data are soft. So long as we retain the conditions of 
complexity and reality, where the kinds and degrees of control which 
can be imposed are limited, this will be the case. Nevertheless, 
this ability to break through the rigid limitations of traditional 
approaches is invaluable in providing fresh insights and a new order 
of analytical data. 

Laboratory games (simulations) focusing on communication and decision 
making have in recent years been conducted for purposes of both 
education and content research. (Nations are represented by teams of 
students who act out the roles of decision-makers; they must make a 
number of decisions affecting their nation’s interest in both domestic 
and foreign affairs.) 

For education, the aim is to give students an appreciation — a feel — for 
the complex and subtle dynamics which operate the real world, which could 
not be conveyed through conventional teaching methods. 

Research use has tended to focus on one of two types of content: policy 

research — to study particular situations; and basic theory, intended to 
develop explanations for real-life behavior phenomena. For example, 

What kinds of communications do decision makers use in crisis situa- 
tions: Under what conditions are threat communications perceived? 

How is the use of communications related to individual personality 
characteristics? 

Most of these simulation efforts have used manual laboratories consist- 
ing of partitioned rooms, chairs, tables, paper forms, means of 
passing written messages among teams (runners) , and on-the-scene 
observers. Accordingly they have suffered fundamental limitations, 
many of which can be overcome with the computer-based laboratory 
concept noted below. For example, a recent large-scale educational 
simulation, not untypical, has been criticized on the basis that it 
was the manual mechanics of the game, rather than the realistic 
communication decision process that claimed players’ attention and 
determined outcomes (leading to the charge that perhaps the wrong 
things were being learned) . 
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In exploring theory content, this type of simulation attempts to 
abstract critical system variables from real life, (based on hypotheses 
and conjectures appearing in standard textbooks) and to model their 
interactions. It emphasizes operational definitions and, where possible, 
the quantification of variables and the systematic recording of data. 
Replication of runs is used to demonstrate the stability of results 
obtained. These simulations have been criticized on a number of well- 
founded grounds, such as the questionable significance of the system 
variables selected and the representation of their relationships; the 
omission of critical situational factors that operate in the real world; 
the use of naive, inexperienced players, driven perhaps by inappropriate 
motivation; the grossly unrealistic manipulation of simulated events 
for experimental purposes. 

Policy-research games attempt to avoid such pitfalls. Rather than 
being general, they tend to be intensive single-case studies of likely 
situations close to reality and involving real countries, as perceived 
by experts. Policy-oriented games typically use realistic political- 
military-geographical settings; they use seasoned political experts 
as decision makers; they give attention to a wide range of relevant 
historical and current events; they use realistic action scenarios— 
the product of expert judgment — to guide the game; they require players 
to prepare national-position papers, which are reviewed by a control 
team of experts, who also manipulate the game in a way judged to be 
realistic for the simulated situation. 

Because the policy-oriented researcher is less concerned with testing 
theory than with gaining an understanding of decision making 
processes in particular situations, and he does not typically attempt 
to subject his detailed data to systematic recording and analysis. 

Data implicit in the player's position papers, the umpiring decisions, 
the player's values, perceptions, motivations, etc., in conjunction 
with given moves and the progress of the game, are not objectively 
assessed; the methodologies are typically unevaluated. 

In summary, these simulations appear to suffer from fundamental limit- 
ations that appear critical for their further development and validity 
for education and research. Neither type of approval produces suffi- 
cient explicit data to allow a detailed micro-analysis of ongoing 
behavior processes that may be essential for understanding the step- 
by-step development of events, the predicted molar relationships, and 
the final results. What is not fully acknowledged is that when a 
simulation game reaches a certain point of complexity, it is no longer 
feasible to attempt to demonstrate patterns of cause and effect 
exclusively by extensive replications of runs. This is true not only 
because of the prohibitively high costs of replication, but because 
of the sizable interrun variability that may be anticipated for even 
the most carefully executed and controlled complex game. 

To combine the assets of these different approaches requires a reformula' 
tion of the data collection and analysis problem. Even where extensive 
replication by itself is a feasible alternative to establish reliability 
and validity of results, it appears necessary to augment the data 
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collected in the standard simulation experiment by emhedding it in an 
extensive matrix of fine-grain observations. Simpler experiments 
have demonstrated that these details of response permit a more adequate 
reconstruction of the perceptions and attitudes as well as the responses 
of the participants, and the interplay of these as they unfold in the 
sequential interactions of events and responses to events. 1 These 
systematically sampled data points allow us to map the micro-processes 
and to demonstrate patterns that underlie the predicted and unpredicted 
mo l ar relationships . To record all these phenomena, and to afford 
experimenter control over them, requires a capability for on-line 
recording and evaluation that only the computer-based laboratory is 
capable of realizing. 

Three distinct uses are made of the computer in this type of simulation: 

a. Computer-administered experiment . The laboratory configuration 

consists of rooms or cubicles which isolate groups of decision 
makers from each other. These groups communicate with each 
other through teletypes, linked through a computer: the 

computer acts as an experimental tool for on-line presenta- 
tion of the situations to the subjects, which may be controlled 
at the discretion of the experimenter through preprogrammed 
actions or on-line manipulations. Each group sends its 
messages, bids, threats, offers, via the interactive console 

to another group which receives it on its own console display 
surface. All such actions are routed through the computer. 

In this way, the computer programs can administer the experi- 
ment-umpiring the legality of moves, providing displays of 
game-relevant information, and recording all moves, messages, 
and times. 

b. Computer-controlled, in-play questioning , to augment multiple 
inquiry data. The answers to many questions on communication- 
decision behavior can be found in the detailed process data 

of moves and countermoves and in the in-play subject reports 
that reveal how communicators shift their evaluations, alter 
their goals, and attempt to induce changes in those they are 
communicating with. A complete analysis decision behavior 
requires not only data on the overt pattern and sequence of 
verbal exchanges, but also requires a parallel assessment of 
the communicator’s intentions, expectations, and perceptions 
of relative status of those he is dealing with. Data are 
also needed on the temporal unfolding of communications and 
on associated shifts in player perception that indicate what 
the communicator intends to do, believes he is doing, plans 
to do — and why. 



^Shure, G. H. , Meeker, R. J., Moore, W. H., Jr., and Kelley, H.H. 
Computer studies of bargaining behavior: The role of threat in bargain- 

ing. SDC document SP-2196. February 9, 1966. 
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In addition to administering the experiment, the computer is 
programmed to aid in the assessment of game moves by collect- 
ing associated subjective data at critical points during the 
bargaining process itself. Between game messages and moves 
or between trials, the computer, through displayed questions, 
can ask the subject to rate or rank various bases of his 
actions, his current intentions and expectations, his 
perception of his opponent, or, through subtle or disguised 
items, it can indirectly elicit the subject’s attitudes 
about the bargaining situation. Thus, an important type of 
behavioral data that is otherwise practically unobtainable 
is gathered, together with the more detailed process data. 

This enables the investigates to identify and pinpoint the 
dynamics of the communication process both at the individual 
level (the reasons and motivations behind certain actions) 
and at the interpersonal level (action-reaction effects). 

c. Data management and analysis . If computer experiments of 
the kind described are conducted "on-line", the amount of 
data the system is capable of collecting can become impossibly 
large. When these data are hierarchical, sequence-ordered, 
and of variable length, the problem of data management becomes 
overwhelming, particularly that aspect of it that attempts 
to identify and classify configurations of moves and to map 
stage-by-stage changes in patterns of use. For the collected 
data to be of maximal inductive value, advanced techniques 
must be used to cope with problems of data scanning, reduc- 
tion, and analysis. 

The foregoing pages have proposed a maximally-ef fective computer-based 
simulation laboratory and have presented illustrations of types of 
projects related to multi-media problems which it could support. The 
question remains, then, as to whether such a laboratory is feasible 
and useful. 

In the restricted sense of technical feasibility, a computer-based 
simulation laboratory for studying multi-media problems is indeed 
feasible. One could be constructed and operated on the basis of existing 
technology. A more important question, however, concerns the value or 
cost-effectiveness of such a laboratory. Taking this broader point 
of view — which can be referred to as " cost-effectiveness feasibility" — 
there are very serious doubts implied by the foregoing discussions as 
to whether this type of laboratory should be recommended. 

As Chapter II made evident, current multi-media problems will not be 
solved merely be developing new techniques to facilitate investigation, 
in the sense of finding ways of doing traditional studies with greater 
finesse. Moreover, even going beyond a simple linear extension of 
current research methods and thus attaining a more comprehensive 
concept of research would still be far from enough. The enormous 
technical apparatus of a computer-based simulation laboratory is not 
what is needed in order to review, analyze, evaluate, and generate 
alternatives to current multi-media practices and the assumptions and 
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institutions in which they are embedded. Obvious problems exist, 
and alternative solutions are already being proposed and — to a limited 
extent — tried out. But even the optimum simulation laboratory described 
above could do little by itself to define and implement solutions to 
multi-media problems. Additional basic research, it may be assumed, 
will always be required. But today there is no specific evidence that 
the great educational problems are awaiting further research results 
before they can be solved. The difficulties which seem critical 
today are those of putting into effect what we already know — of making 
educational goals explicit, and of caring enough to reconstruct 
educational practices and institutions so that they effectively reach 
these goals. 

In the terms of the Guba-Clark model, the problems of development and 
diffusion are currently of greater importance than those in the research 
area. It may even be a mistake to claim that research is of primary 
long-run importance, since (as has been repeatedly demonstrated in the 
past) research conducted in isolation from development and diffusion 
tends to be inconclusive, difficult to apply, and generally of doubtful 
value. For these reasons, the simulation laboratory approach to 
multi-media problems appears infeasible, in the sense that such a 
laboratory could do little by itself toward reaching major problem- 
solutions. 
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IV . Feasibility of a Comprehensive Development 
Laboratory for Multi-Media 



A. Concept of a Comprehensive Development Laboratory 

As was indicated at the beginning of Chapter III above, the concept of 
the "comprehensive development laboratory" has as its distinguishing 
characteristic that it is concerned with integrated and continuous 
planning, supervising, evaluating, and supporting systematic educational 
improvements. As such, it combines in a coherent fashion aspects of 
each of the other five types of laboratories: planning and policy 

formulation, basic research, component development, simulation, and 
experimental schools. 

No laboratory of this comprehensive sort as yet exists — at least in the 
full degree envisaged by this report. Educational Facilities Labora- 
tories, established by the Ford Foundation "to help schools and colleges 
solve their physical problems by encouraging research, experimentation, 
and the dissemination of knowledge regarding educational facilities," 
is perhaps the closest approximation to the concept. EFL, however, 
does not emphasize systematic operations to the extent of ensuring 
fully integrated planning - development - evaluation activities. 

Some of the Regional Educational Laboratories which have been established 
by special legislation also approximate to some degree the concept of a 
comprehensive development laboratory. None of these organizations, how- 
ever, has undertaken to systematically attack the complete spectrum of 
problems related to the design, development, and implementation of 
major multi-media and/or telemedia innovations. There is reason to 
believe that the laboratories migHt"* reconsider their abilities to deal 
with such problems, but their primary contribution appears to lie in 
other areas. The kinds of problems envisioned for a comprehensive multi- 
media development laboratory will require a concentration of attention, 
and a freedom from pressures for short-term visible results, which do 
not presently tend to be characteristic of the Regional Laboratories. 

The general concept of the Comprehensive Development Laboratory is 
illustrated in part by Figure 1 in Chapter II above, "Schematic of 
Complex Man - Machine System Development Process." This figure indicates 
the interrelations which are necessary among the various phases of system 
and sub-system planning, design, description, specification, production, 
installation and operation. Overall needs for "technical management and 
design control" and for "evaluation and evolution" are stated. These 
are brief reminders of vitally important system principles which were 
learned the hard way — by violating them or by ignoring them. Too 
many complex systems have been built and have failed (or have needed 
major reconstruction) because of insufficient attention to advanced 
planning (analyzing system goals, environmental constraints, and 
available resources), to specification of hardware - software - personnel 
functions and interfaces, to orientation and training, to testing sub- 
systems and the total system against stated goals, and to the need for 
evolving as new or unanticipated problems and opportunities arise. 
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Figure 1 implies research activities, and least in the sense of applied 
research, in connection with prototype developments which are usually 
advisable at various times during complex system development. Basic 
research is generally of a longer-range nature, and would not be attempted 
as an actual part of the process of developing a specific system. Experi- 
ments with parts, aspects, or simulations of a system under development 
are highly appropriate and necessary, however. They should be planned 
for and carefully managed, to insure genuine contribution to solving 
developmental problems. Applied research in a broader sense, including 
operations research and environmental analysis, is also pertinent. 

Diffusion and dissemination are also here intended as essential func- 
tions of the comprehensive development laboratory. In this respect, 

Figure 1 is inadequate and needs to be supplemented by the Guba-Clark 
model, as well as by indications of strategies and tactics for success- 
fully sustaining operations in an increasingly wide environment. 

The concept of the comprehensive development laboratory, then, contains 
all the foregoing elements, plus an overriding commitment to a total 
approach which will develop optimum multi-media applications within the 
context of a genuinely effective — and improving — educational system. 

B. Implications for Organization. Facilities and Personnel 

1. Organization . The comprehensive development laboratory concept 
outlined above has definite Implications for the types of activities in 
which the laboratory would and would not engage. Implications for 
laboratory organization are closely related, since organizational 
structure should facilitate and reinforce laboratory tasks. Figure 2 
indicates a possible functional organization based on three fundamental 
types of activities — plans, operations, and evaluation. In addition, 
a "meta-development" activity is required in order to bring the 
laboratory into effective existence; various support activities are also 
necessary. 

The organization illustrated in Figure 1 contains a number of functions 
which are separately important and which must not be overlooked. It is 
equally important that no one of them be permitted to dominate the others, 
as often happens in organizations. Planning, operations, and evaluation 
are all indispensable; but generally speaking, it is unwise to give 
operators the sole responsibility for planning or evaluating their own 
activities. Similar comments apply to the possible dangers of permitting 
either planning or evaluation to be undertaken for their own sake, with- 
out being thoroughly integrated into ongoing operations. 

From another viewpoint, the organization of the same laboratory could 
distinguish those operations directly related to multi-media equipment 
from those which are more conceptual, reflective, or analytic in nature. 

In that case, a "research and development laboratory" could be distinguish- 
ed from an "institute of advanced studies". This distinction, however, 
would be inappropriate for the laboratory here conceived, which emphasizes 
the integration rather than the separation of thought and action. 
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Figure 2 . Comprehensive Development Laboratory Organization 



2. Fac .lities. The organization here presented will plan multi- 
media applications, directly or indirectly carry them out, and evaluate 
them as contributions to widespread improvements in education. At the 
outset, it is doubtful that such a program would require extensive 
physical facilities. Indeed, it would be safer to begin with the 
assumption that physical facilities other than office space are not 
needed, to emphasize that the success of the organization is to be 
judged by its impact on education, not by the magnitude or expense of 
its inventory and arrangements. 

Without experimental facilities of its own, without a computer , f 'and 
without a demonstration school to operate, such a laboratory can never- 
theless be in a strong position to organize and direct a fruitful and 
influential program of activities to be carried out in a variety of 
schools and research centers. But of course the laboratory must be 
well-managed, imaginatively directed, and adequately funded. It must 
have a strong, capable staff, dedicated to its purposes and techniques, 
and intimately involved in all details of the activities it sponsors. 

It must have access in a variety of ways to on going school programs 
for purposes of observation, trial, and evaluation of designs. All 
sponsored activities must be well-planned and clearly defined as integral 
parts of a coherent program. 

It is possible that as the multi-media laboratory develops its programs, 
it will identify tasks which can most effectively be carried out direct- 
ly by the laboratory's own staff. If so, it may acquire experimental 
facilities of its own. These, however, should be made to depend on 
later analysis and assessment, and should not be included in original 
planning. Such facilities would vary with particular programs of 
research, but might eventually include some or all of the following: 

Menus of multi-media devices, equipment, and support facilities 

Resources for the development of experimental software and other 
programs for the multi-media devices 

Sufficient computer capability to permit the design of simulations, 
games, and other applications 

- Facilities and equipment for "walk-through" simulations and related 
techniques 

In the pre-equipment stage of the laboratory (which may continue in- 
definitely), geographic location can be based on a comparatively limited 
set of factors of convenience, including the availability of research 
personnel, cooperative school systems and other educational organiza- 
tions, and related research and development efforts. Proximity to a 
major center of air transportation would be of interest. 

3. Personnel . The basic structure of the laboratory would evolve 
around a team organized for the systematic development of programs and 
tasks. Although various activities might require special staffing, the 
basic laboratory staff would include permanent expertise in the following 
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areas : 



- Educational systems analysis, design, and development 

- Multi-media equipment design and operation 

- Multi-media materials development and production 

- Engineering and design of learning spaces 

- Learning theory, concept organization, and child psychology 

- Testing, evaluation, and psychometrics 

- Modeling, simulation, and gaming 

- Educational management, economics, and politics 

- Cost-effectiveness analysis 

This group would constitute an interdisciplinary team who would find 
their common theme in the integration of their various specialties 
around major problems involving the design and implementation of multi- 
media systems. Each of these groups would analyze needs in its own 
area and would be free to recommend needed projects. Although skills 
devoted to major problems would change according to the problem situation, 
the essential task would remain the design, evaluation, and eventual 
implementation of complete educational systems optimally incorporating 
media. 

C. Functions 



A comprehensive multi-media development laboratory conceived along the 
lines indicated above would be engaged in a broad spectrum of functions. 
The general mission of the multi— media laboratory would be to develop, 
test, and actively disseminate systematic cost-effective uses of tele- 
communications, media, and other technological innovations for educa- 
tional purposes, in such a way as to begin and sustain changes of major 
significance throughout American education. In fulfillment of this 
mission the multi-media laboratory should devote its attention to: 

- Problems of system design and implementation 

- Measurement of alternatives 

- The development of implementation models 

- The development of improved information systems and feedback 

- The recording of replicable experiences 

- The assistance of interested groups in mounting and evaluating 
multi -media instructional programs 
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- The dissemination and demonstration of worthwhile prototypes 
Comments on aspects of these functions follow: 

Operational Testing and Development . One of the most important functions 
required is developing and testing designs which embed instructional 
media and techniques in operational teaching— learning environments. 

This would permit the examination of technology utilization within a 
context of complexity of the actual environments in which technology 
must be used, i.e., as part of a total instructional system. It would 
be possible to identify those factors that complement, enhance, or 
hinder the effectiveness of the technology; and required modifications 
to media and other system components can be determined. By making the 
required modifications and engineering the total system as dictated 
by evaluated experience, the advantages and limitations of technology 
can be determined and the necessary conditions for its appropriate 
utilization specified. The objective would not be to perform compara- 
tive evaluations of the effectiveness of competing media, but to test 
and develop, by iterative cut-and-fit procedures, media and other 
components of the total instructional system. The end product would be 
demonstrably effective instructional media and techniques with docu- 
mented specifications for their appropriate utilization in the school 
environment . 

Empirical Research . A laboratory, in a more conventional sense of the 
term, may be used to perform studies aimed at acquiring new knowledge 
through direct observation of the behavioral results of technology 
utilization in learning situations. It is useful in considering the 
specific focus of the laboratory to distinguish between experimental 
and exploratory studies. Experimentation and exploration are distingish- 
ed here on the basis of the extent to which the observation and data 
recording and analysis are guided by precise hypotheses and the degree 
to which rigid controls are exerted. 

• Experimental Studies . Some problems of educational technology may 
be amenable to rigorous experimentation, i.e., the formulation of 
testable hypotheses, the controlled manipulation of prescribed 
variables, and the measurement of concomitant variation in other 
variables. Such research requires formal theory to guide the 
formulation of hypotheses. Since it is doubtful that adequate 
theory exists for classroom learning in general or for educational 
media in particular, it will usually be necessary to resort to 
less formal techniques of empirical exploration. 

• Exploratory Studies . Even where understanding of a given problem 
is meager, that problem may profitably be represented in a 
laboratory or examined in the field for purposes of direct experi- 
ence and exploratory observation. This approach is appropriate 
when there is inadequate theory to identify critical relationships 
and generate testable hypotheses. However, orderly search can make 
important contributions to the clarification of problems and to the 
stimulation of new insights and hunches that may lead the way to 
more formal experimentation or systematic trial. 
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Analytic Research . A computer facility within a laboratory could 
contribute directly to analytic research, not only through straightfor- 
ward computational support, but also through symbolic simulation. In 
order to deal simultaneously with a large number of intricately related 
variables, it may be possible to develop analytic models of those 
variables and their interrelationships which may then serve as the 
basis for computerized simulations. By this means it may be possible 
to approximate solutions to otherwise unsolvable problems through 
artificial sampling and other solution-by-analogy techniques. However, 
computer studies within a laboratory should themselves be carefully 
planned and evaluated. There have been far too many instances where 
computer operations have absorbed a disproportionate share of research 
resources, yielding results of little if any superiority to those 
derivable by less elaborate means. 

Demonstration . A laboratory can serve as an effective means of two- 
way communication between researchers and educational practitioners. 

It can provide the capability for live demonstration of the results and 
products of a technology research and development program, which in 
turn will provide direct experience to experts throughout the educa- 
tional community as a basis for critically examining demonstrated re- 
sults and posing new problems. This may well be the most effective means 
for promoting increased participation and expert criticism in educa- 
tional research and development. This increased involvement may lead 
to improved user acceptance, and should accelerate the transition of 
technological advances to classroom application. 

Integration . A laboratory should provide the opportunity to integrate 
the problems, results, and methods of other research and development 
activities. Where related problems or educational elements have been 
isolated for purposes of study and analysis, these problems and separ- 
ate study results may be related and combined. There may also be 
advantages to be gained in integrating not only the content or 
subjec matter of the research, but also the research methods of them- 
selves. For example, the use of classic experimental techniques with 
analytic simulation in some iterative combination may provide a power, 
scope, or efficiency not possessed by separate methods alone. The 
laboratory, by providing common tools and informational bases, at 
least supplies the opportunity for concerted, cooperative attack on 
significant problems by a variety of analytic and empirical methods. 

D. Techniques 

An important aspect of the comprehensive development laboratory will be 
its conscious and explicit subordination of techniques to problems. For 
this reason, it will avoid commitment to a limited set of favored 
techniques. It should be prepared, on the other hand, to use any 
techniques which may be required and which are Justifiable as probably 
yielding returns commensurate with their demands on the laboratory's 
resources in funds, people, and equipment. 

Simulation techniques , for example, have been discussed in the previous 
chapter. Although they did not appear promising enough to justify a 
laboratory devoted exclusively or primarily to their use, they should 
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be considered as among the possible tools for a comprehensive develop- 
ment laboratory. Since the laboratory will have no predetermined 
commitment to simulation, balanced judgments should be possible on the 
value of simple or complex simulations in particular cases. The status 
of an issue may be such that simulation is not the right tool. Some 
problems are too vague for rigorous experimental treatment and require 
preliminary definition by means of field research or other research 
methods. Some problems are so highly specific, on the other hand, that 
simulation is too elaborate. Methods such as small scale experimenta- 
tion or graphic analysis should then be considered. 

Similar comments apply to digital computer techniques . The compu-er 
nas made possible vast new potentials for the understanding of educa- 
tional behavior through its ability to deal with tremendous amounts 
of data at fantastic speeds. In fact, so great is the computer’s 
potential for dealing with mass data, that its abilities have often 
overreached the current state of analytical sophistication. 

Any consideration of new laboratory techniques must include the computer 
as a possible analytical tool, but the critical issue at this juncture 
is the determination of its appropriate use. The computer functions in 
social science research in one of three ways: (1) it analyzes data and 
performs statistical computations according to a predetermined format; 
(2) it organizes data and determines appropriate computations based on 
prescribed conditions; and (3) it extrapolates and predicts trends and 
future conditions according to interrelationships among inputs. The 
computer is, in short, a computational tool, an analytical tool, and 
a problem-solving tool. Its application to research in education, how- 
ever, depends on the "state-of-the-art" of the efforts which support 
these applications. 

Often, a computer's principal influence on a process or an investigation 
has been through the impetus it has given to careful analysis and 
evaluation. Since computers are expensive and precise, they tend to 
demand unusually painstaking formulations of issues and of results. It 
then frequently turns out that although clarification has been achieved 
and decisions have been improved, the computer in itself made no 
contribution, since its actual computations were either self-evidently 
true, or irrelevant, or derivable by other and simpler means. A multi- 
media laboratory should avoid wasting its resources on elaborate 
computation, when the actual requirement is for penetrating analysis. 

In general, a developmental, problem-solving laboratory should be 
prepared to use and defend less formal methods than those which are 
usually associated with the idea of laboratory research. This will not 
be easy, since wide knowledge and considerable good sense will have to 
be substituted for methodological formulas. Traditional experimental 
methods will not often produce results having real impact on anything 
so complex as public education. In order to achieve the precision and 
control required by traditional methods, the phenomena of real interest 
are usually stripped out of the problem. The result is often a precise, 
unambiguous answer to a question that nobody has asked. 
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In order to obviate the difficulties of traditional experimental 
methodologies, and at the same time to avoid the pitfalls of new yet 
incompletely developed techniques, the following methods appear useful 
for the field of education: 

Cut-and-Fit Methodology 

This method requires establishment of behavioral objectives and a 
process of try, fail, try, succeed. The "success" may not be optimal 
and its causes are not always clear, but the process allows the elimina- 
tion of the ineffective and some insight into the degree of change 
necessary within a situation. First trials should be small-scale and 
short-term, gradually building into more complete attempts. 

Problem Analysis 

A good research strategy for studying problems occurring in large, 
complex systems is initially to embed — really or symbolically — 
the system of interest in a noiseless environment, i.e., one which is 
completely deterministic. After having determined how the system be- 
haves in this environment, disturbing elements can be added o**- at a 
time to determine their effect on the system’s behavior. lnJ i p: .„<• ire 

will not isolate all of the possible interaction effects or in general 
satisfy the requirements of ideal research design. But it is a systema- 
tic way to deal with very complex systems and does permit the addressing 
of problems that are unmanageable by traditional pure methods. 

Informal Simulation 



Informal simulations of some aspects of the educational situation are- 
feasible and can contribute to educational improvement. Enough is 
known about student behavior so that its limits can be specified and 
the interesting points on a continuum can be identified. These and 
their assumed relationships to other educational factors can be in- 
corporated in a model which can then be used in sensitivity studies. 

Such studies will not uncover new knowledge, but they can indicate where 
the potential pay-offs lie and guide subsequent empirical research and 
development. 

Walk-through simulation is another promising technique for multi-media 
development. This consists of analyzing some complex process and then 
having a group of technicians "walk-through" or role-play the steps 
or events in the process. It can provide important insights and avoid 
mistakes prior to a full-blown field test. Walk- through simulations of 
this type are similar to new developments in micro- teaching, and offer 
great promise as relatively inexpensive, uncomplex research devices. 

Delphi Techniques 



The Delphi Method [63] has as its purpose the utilization of expert 
opinion in situations which lack a theoretically convincing reason for 
selecting a particular action or a particular policy for action. It 
replaces direct debate by sequential individual interrogations inter- 
spersed with information and opinion feedback from earlier parts of a 
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program. The methodology can have important consequences for educa- 
tional research, especially as concerns the complex situations in which 
multi-media operate. A careful analysis of expert opinion across the 
many subject fields and specialties involved in the design and develop- 
ment of the learning situation could help to resolve both major and 
minor implementation problems. 

Educational Planning Games 

Great interest has been expressed recently in the use of games for 
educational purposes, and an educational planning game has been develop- 
ed by Clark C. Abt. [See in 66.] It uses four teams of educational 
experts, two of which represent groups of professional personnel, one 
represents community interests, and the fourth represents "inhibitors" 
or reality screens . The first two teams develop a plan or strategy 
for attacking a significant educational problem and are then confronted 
by the countervailing forces represented by the other groups. The 
result, if the game runs long enough, is a compromise position which 
takes into account all of the factors pressing upon an issue. In this 
manner the participants recognize the forces acting in a situation 
and are able to develop a realistic solution. 

The Delphi Method and Educational Planning Games hold significant 
potential for research in multi-media and telemedia and their applica- 
tion to educational situations. Both approaches permit the necessary 
conjecturing about changes in present systems, and they allow a full 
consideration of the forces affecting a specific problem. 



E. Programs 



A gr^ah variety of activities would be appropriate for a comprehensive 
development laboratory with the structure and functions described 
above. However, it would not direct its energies primarily toward 
invent! ig or developing particular multi-media devices; producing 
software packages, instructional packages, or curriculum packages— 
except on a prototype basis; training personnel in large numbers; 
building or operating a school or institution for a long-range program 
of education; or carrying out controlled, experimental activities 
aimed at basic research or the creation of knowledge. Its programs 
of course might result in suggestions for any of the above activities, 
and in such cases recommendations would be made that they be supported 
by other agencies. 

The laboratory^ basic concerns would be with working out and demon- 
strating systematic sets of solutions to important educational problems. 
It would focus its efforts on particular tasks which could be formulat- 
ed in terms of evaluatable goals, and which would give reasonable 
promise of contributing significantly to the advancement of education. 
The laboratory would intend to make a major difference in the conduct 
and results of education, not just a difference in theoretical under- 
standing or in superficial appearance. 
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Sample program areas consonant wit u the laboratory's concerns and 
methods would, at a general level, Include the following: 

- Improved learning of particular subject matters. 

- Increased range of student-to-student Interactions for learning 
purposes . 

- Improved educational opportunities for isolated or scattered 
populations. 

- Development of criteria for mix-ratios of professionals, para- 
professionals, and technology within the present context of the 
school so as to effect cost savings and educational improvements. 

- Practicable models of improved learning environments and of alternative 
configurations of elements. 

More specifically, project area? such as those referred to in Section 
III.F above (Feasibility of a Compute l -B ased Multi-Media Simulation 
Laboratory) would be candidates for defining comprehensive development 
laboxatory tasks. There would be significant differences in emphasis 
and treatment, however, as indicated in the following comments. 

1. Education of the Handicapped . The simulation techniques which 
were described in III.F in connection with the social education of the 
blind were partly for instructional purposes, partly for observation 
and analysis. That is, simulation was both a teaching vehicle and a 
laboratory control vehicle. It was hypothesized that computer techniques 
could be used to generate and evaluate social learning experiences 
for blind children. The simulation laboratory would conduct experi- 
ments to evaluate the merits of this hypothesis. 

A comprehensive multi-media development laboratory concerned with this 
problem area would not take the same approach. It would instead attempt 
to evolve a description of a complete instructional system for the 
blind, identify possible multi-media contributions to such a system, 
and enable operational trials to take place and be evaluated in realistic 
environments. These activities of course would necessitate the 
clarification of educational goods for the blind, a review of present 
systems of attaining those goals, generation of feasible alternative 
systems, analysis of cost-effectiveness trade-offs, simulated and/or 
prototype trials, and sustained commitment to evaluations and 
revisions as needed and justified to improve system performance. 

This comprehensive task would be conceptually much larger than the 
previously described computer simulation task and should have more 
significant results, but its costs should be no greater — and perhaps 
less. The principal effort would go into comprehensive analysis, 
design, and trial, rather than into concentration on particular 
equipment or techniques. 




62 



2. Education In the Sciences . Similar comments apply to this 
problem area. The computer-based approach to scientific inquiry 
previously described in Section III.F. may have merit. However, it 
is risky to embark on an experimental investigation centered around 
this computer-based approach, partly because of the costs which 
would necessarily be involved, but also — and more importantly — because 
once a commitment is made to a computer-based experiment it is diffi- 
cult for the experimenters engaged in it to be objective about the 
value of the experimental systems they devise. The commitment of the 
comprehensive development laboratory, on the other hand, is not to 
investigation of a particular hypothesis or technique, but to imagina- 
tive generation and careful comparison of alternatives. Objectivity 
therefore tends to be encouraged and reinforced; it is also facilitated 
by the comprehensive laboratory's organizational separations between 
plans, operations, and evaluation. 

3. Education of the Culturally Disadvantaged . Reference to the 
description in Section III.F. of experimental work in connection with 
the education of the culturally disadvantaged will once more reflect 
the prevailing overemphasis on technique. The question implied is, 

"Will this technique work?" rather than "What is the role, if any, of 
this technique in reaching an optimum solution to the educational 
problem?" There is a profound difference between these two questions, 
and the methods for answering them are also very different. To start 
answering "Will this technique work?" implies a commitment to the 
technique and then repeated trials to see what can be done with it. 

To start answering "What is the role, if any, of this technique in 
reaching an optimum solution to the educational problem?" implies 
deferring trials of the technique until goals and alternatives are 
well-understood, and then trying out the most promising alternatives. 

4. Means of Extending Teacher Effectiveness . The discussion 
in the preceeding chapter of the "enabling" approach to multi-media 
(and thence to teacher effectiveness in general) represents a 
significant but incomplete step toward the comprehensive development 
laboratory. It recognizes the need for doing more than merely developing 
and testing multi-media devices and materials in isolation. However, it 
suggests only one favored method for improving the situation (teacher 
participation in media package design and construction), rather than 
insisting on the need for a comprehensive approach to the total problem 
of teacher effectiveness. 

5 . Communications/Decision-Making in International Relationships . 
The simulations described under this heading in the previous chapter 
represent highly specialized research activities and/or complex, costly 
educational experiments. In either of these aspects, these simulations 
would be inappropriate for a comprehensive multi-media development 
laboratory. By their nature, they embody sustained exploration of a 
technique to which a commitment has already been made. No claim is 
made or intendea that their starting motivation was from a recognized 
educational problem, or that the educational techniques being explored 
are to be systematically and objectively weighed against alternatives. 

In this connection, it should be pointed out that the comprehensive 
development laboratory would never allow its consideration of techniques 



to begin with expensive commitments. It is entirely possible, of course, 
that highly complex and expensive approaches to educational problems may 
eventually be developed and defended on the basis of their high returns 
in educational effectiveness. Indeed, the comprehensive development 
laboratory should actively generate, collect, and investigate ideas for 
complex problem-solutions. But its investigation will begin with paper 
plans and conceptual trials, and then move through limited and compara- 
tively simple tests, before embarking on full-scale system construction 
and operation. 



F. Difficulties 



Establishing a comprehensive development laboratory alonp the lines 
described above will not be easy. Many of the difficulties, though, 
will be more or less routine, in the sense that they could be expected 
in connection with any new laboratory or organization. These aie such 
matters as securing adequate funding, acquiring an appropriate staff, 
and building up staff agreement on the significance of laboratory policies 
and methods. 

Other difficulties which may be anticipated are present or potentially 
present whenever educational changes are tried out. Because of the 
fears which prospective changes always tend to create, the sensitivity 
of the field of education, and justifiable concern for the effects of 
research techniques and new developments on children, strong constraints 
may be placed on experimentation. 

Special difficulties with a comprehensive development laboratory may be 
anticipated because educational situations, including those to which 
multi-media can make a contribution, rarely permit the explication of 
a clear-cut statement of aims and goals broad enough to account for the 
total educational experience, especially over a considerable period of 
time. While this complaint is often heard, and frequently countered 
by pointing to the development of "behavioral objectives" (or arguing 
that education is an art anyway) , much remains unaccounted for in any 
deliberate educational experience. Frequently, any one experience will 
contain a number of incompatible goal systems operating within the same 
environment. This confusion of educational purposes makes output 
analysis difficult. 

In addition, only limited information is available on the inter-relations 
among significant elements in a total educational situation. There has 
been considerable research on parts of the problem, including the adapta- 
tion of mediated instruction, but there has been insufficient attention 
to the nature of the total situation and the effects of varying one or 
more elements. 

Other special difficulties would be caused by the facts that many of the 
techniques referred to in Section D above are in early stages of 
development, and that personnel who understand their uses and limitations 
are in very short supply. 
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The existence of these several types of difficulties, however, does not 
mean that a comprehensive development laboratory must be deferred until 
all difficulties cease to exist. On the contrary, it is implied by the 
laboratory’s concept and methodology that among the laboratory’s functions 
will bs clarification of educational goals, analysis of interactions 
among elements in total educational systems, improvement of its own 
techniques, and training of its own staff. All laboratories engage in 
shaping their own conceptual tools and in developing their own personnel. 
The comprehensive development, laboratory is committed to iteration and 
evolution. Its participants will learn (through actual trials, through 
evaluation of successes and failures, and through progressive improve- 
ments) both how to think about and operate the laboratory itself, and 
how to improve education. 



G. Feasibility . 

A comprehensive multi-media development laboratory of the sort 
described in this chapter is needed, is technically feasible, and would 
have an excellent probability of making significant contributions to 
education without excessive costs. The preceding discussions have 
pointed out that "multi-media problems" exist, and that opportunities 
for fully effective educational uses of media are unlikely to be 
realized unless a much more comprehensive and systematic approach is 
taken than any that is being used today. There is no technical reason 
whatsoever to prevent a comprehensive multi-media laboratory from 
coming into existence and operating successfully. The difficulties 
would be those of inertia, institutional resistance, and a lack of 
support. 



Financial support is the ultimate key to the feasibility of the 
comprehensive multi-media development laboratory. With adequate support, 
there is good probability that the laboratory would rather soon demon- 
strate its value and begin to establish the credibility of its methods. 
However, support should include sufficient guarantees of stability to 

ensure that there would be no undue pressure for quick results , and that 
well-qualified personnel would be attracted to participate in it. Annual 
costs of around $2,000,000 are suggested by the following rough 
estimates: 



25 full-time professional staff at an average 
of $16,000 

Non-professional support, office space and 
equipment, publications, travel, and employee 
benefits 

Field operation and evaluation (equipment, 
materials, and services) 



ANNUAL TOTAL: 



$ 400,000 



600,000 

1,000,000 

$2,000,000 
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An adequately funded laboratory of the sort described should be able 
to make major contributions to the development and diffusion of improved 
educational systems. In fact, if such a laboratory can not make signifi- 
cant contributions, then nothing else can, except perhaps through 
accidental or chance occurrences. The laboratory as defined would have 
a clear mission, effective methods, and resources and personnel capable 
of sustaining its efforts through the stages of discovering and design- 
ing what is needed, trying out and improving its designs until they 
can accomplish their goals, and catalyzing their widespread dissemination. 



H. Implementation 

There are several ways in which implementation of a comprehensive multi- 
media development laboratory could be carried out. For example, the 
laboratory could be established as an entirely new organization, or 
it could be based on some related existing organization which would be 
given a new direction and charter. The present project, however, has 
not attempted to review existing laboratories, schools, or other organiza- 
tions which would be possible candidates for undertaking to carry out the 
mission of a comprehensive multi-media development laboratory. This would 
be a matter for early attention once the implementation of the laboratory 
is seriously intended. 

Some general observations on implementation methodology follow, along 
with implications noted for the Commission on Instructional Technology 
recently formed as a result of Title III of the Public Broadcasting Act 
of 1967. 



I . Methodology for Laboratory Implementation . As is indicated 
earlier in this chapter in Figure 2 (Comprehensive Development Labora- 
tory Organization), a "meta-development 11 activity is required in order 
to develop the laboratory itself, in distinction from the educational 
systems the laboratory will eventually be concerned with developing. 

Figure 2 refers to meta-developmenc as involving a "pattern of con- 
ceptualization, definition, acquisition, and modification." This is a 
reference to a methodology worked out to a rather high degree of 
sophistication by the U. S. Air Force Systems Command, to assist in 
managing the implementation of complex man-machine systems. The intention 
here is not to suggest that this Air Force system should be followed 
literally and in detail as a methodology for implementing an educational 
laboratory, since very important differences obviously exist between 
educational systems and electronic systems. Nevertheless, explicit 
management techniques have been found extremely useful in developing 
large and complex systems, and there are good reasons to believe that 
they can also be useful (if properly handled) in developing comparatively 
simple educational systems (such as a laboratory), as well as complete 
large-scale educational systems (such as, for example, a network of 
educational cooperatives) . 
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Systems management is the process of planning, organizing, and controll- 
ing the activities of participating organizations to accomplish objectives 
of tim3-phased system programs which require the combined efforts of 
diverse: functional agencies. The process is relevant to the compre- 
hensive multi-media development laboratory both internally (with respect 
to its various organizational divisions) and externally (because of its 
commitments to field trials, evaluation, and dissemination). 

As applied to the process of developing a system, the system mission, 
functions, and configuration are established in the so-called "Conceptual 
Phase". The present feasibility study is a contribution to that phase. 
Next should come the Definition Phase, in which desired accomplishments 
(objectives or "end items") are specified in greater degree; the Acquisi- 
tion Phase, in which the laboratory staff and organization are more fully 
specified, then assembled and evaluated as potentially contributing to 
desired accomplishments; and an ongoing Operations-and-Modification 
Phase, in which improvements are suggested, implemented, and evaluated 
on the basis of actual experience. In the Acquisition Phase, of course, 
considerable attention is paid to analyzing, identifying, and acquiring 
the resources necessary and justifiable for attaining objectives. 



An "adaptive" methodology is implied by the foregoing remarks, as well as 
by discussions earlier in this report of the techniques the laboratory it- 
self should use in developing effective multi-media applications. That is, 
explicit designs and plans are constructed, but it is expected that they 
will need continual review and revision as experience is gained during 
their implementation. No one yet knows enough about educational technology 
to be able, merely by reviewing past experience, to write complete and 
ideal specifications for a comprehensive multi-media laboratory. What is 
needed (at two levels — both for developing the laboratory, and for the 
laboratory itself in developing multi-media educational systems) is a 
systematic program of planning, thought, tentative decision, trial, 
evaluation, and revision. Full-scale trials are usually comparatively 
late, and are preceded by simulated, prototype, or small scale trials. 
Adaptive methodology is required in order to increase chances of success 
in approaching situations which are not completely understood. Educational 
systems eminently represent such situations, and they eminently require an 
adaptive methodology for their design and improvement. 

2. Implications for the Commission on Instructional Technology . 

A Commission on Instructional Technology has recently been formed as a 
result of Title III of the Public Broadcasting Act of 1967. This 
commission is charged with conducting a comprehensive study of instruc- 
tional television and radio, including relationships with other media. 

If the commission wishes to consider methods of improving the educational 
effectiveness of instructional television and radio, the concept of the 
comprehensive development laboratory as outlined in the present report 
is recommended as highly relevant. It is very unlikely that instructional 
television and radio will achieve anything like their true educational 
potential if they are developed and implemented unsystematically, with- 
out proper attention to total educational contexts and purposes, to 
short and long range planning, and to comprehensive evaluation. The 
laboratory described in this report could make a major contribution to 
the design and implementation of complete educational systems making 
optimum use of instructional television and radio. 
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It is probably true that a comprehensive development approach to 
instructional television and radio would not satisfy enthusiasts who 
are already committed to the general value of telemedia. Starting 
from that commitment, the important questions have been how to secure 
additional funds, how to develop new applications, and how to improve 
quality— not how to define an optimum role in a total educational 
context. But if the Commission finds, as this report has done, that 
educational telemedia are at present generally quite ineffective, it 
may also conclude that a new overall approach to telemedia is desirable, 
and that an organization along the lines of the comprehensive develop- 
ment laboratory could combine — as no other organization could all the 
factors necessary for a complete approach. 

If the Commission agrees, its next step would be to recommend that the 
laboratory be more fully defined and specified, and that it should 
then be implemented. 



V. Conclusions and Recommendations 



Conclusions . This study has found that present multi-media uses are 
generally rather ineffective, and that research and development efforts 
in the field are limited in scope, either in their selection of media 
or in their definition of the educational situation of which the media 
are a part. These facts indicate that a comprehensive multi-media 
laboratory, designed to deal with educational problems involving the use 
of multi-media (and tele-media), is needed; that it could contribute 
significantly to the improved implementation of media in present school 
situations; and that it could be of important significance in the design 
of future learning situations. These conclusions are amplified in the 
following paragraphs: 

Problem Locations (Section II. A). "Multi-media problems" neither exist 
nor can be solved in isolation from the total educational system, in- 
cluding its environment. A laboratory to address these problems should 
be concerned with a total spectrum of activities, including research, 
planning, design, operation, evaluation, and diffusion. 



Current Multi-Media Practices and Research (Section II. B). The 
fundamental overall need is for systematic development, evaluation, and 
demonstration of multi-media uses in real situations for actual educa- 
tional purposes. In comparison with what is required, present efforts 
are partial, sporadic, fragmented, and inconclusive. 

Some Critical N e eds for Analysis. Development, and Diffusion (Section 
II. C). In ordt. to make fruitful use of the tremendous educational 
potential offered by new media, new kinds of investigation and develop- 
ment are needed. 



Types of Laboratories (Section III. A). Six types of laboratories can be 
distinguished as being able to deal in various ways with educational 
media. Of these, simulation laboratories and "comprehensive develop- 
ment" laboratories offer greatest promise of needed assistance. 

Feasibility of Simulation Laboratory Approaches (Sections III.B, C, D, 

E, F). So far, simulation laboratory approaches to educational problems 
have shown limited success in relation to costs. A semi -manual, computer- 
based laboratory utilizing time-sharing techniques would apparently offer 
maximum effectiveness for simulation experiments. Such a laboratory is 
technically feasible (that is, it could be constructed), but its main 
contribution would be in the research area. It could not adequately 
review, analyze, evaluate, and generate alternatives to current multi- 
media practices and to the assumptions and institutions in which they 
are embedded. It could do little by itself toward reaching solutions 
to the major problems which lie in the areas of development and diffusion. 
It therefore does not represent a feasible approach to multi-media 
problems. 
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Concept of a Comprehensive Development Laboratory (Sections IV. A, B, 

C, D, E). A comprehensive development laboratory, which would be 
distinguished by planning, supervising, evaluating, and supporting 
systematic educational improvements in an integrated and continuous 
fashion, does not yet exist. If one were to be established with an 
emphasis on multi-media, it should be organized around methodology 
and functions, rather than around equipment and facilities. Techniques 
would in all cases be subordinated to problems. 

Feasibility of a Comprehensive Multi-Media Development Laboratory 
(Sections IV. F, G) . The laboratory as described is needed, is 
technically feasible, and would have an excellent probability of 
making significant contributions to education without excessive costs. 

Recommendations . The foregoing conclusions lead to the recommendation 
that a laboratory should be established and adequately supported, 
broadly conceived to design, evaluate, and influence the implementation 
of complete educational systems making optimum use of multi-media and 
telemedia. An adaptive methodology should be used for its implementa- 
tion as well as for its activities. 



Implications for the Commission on Instructional Technology (Section 
IV. H. 2). It is recommended that the Commission on Instructional 
Technology, formed as a result of Title III of the Public Broadcasting 
Act of 1967, consider the relevance of a comprehensive multi-media 
development laboratory to the problem of improving the educational 
effectiveness of instructional television and radio. 
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Appendix B. Contacts and Visits 



The project staff attempted to reach a broad spectrum of opinion and 
information on the present and progressing state of technological 
practices by contacting and visiting as wide a range of individuals and 
institutions as time permitted. These meetings were generally informal, 
since the technical reports and publicity information supplied by most 
institutions contained adequate descriptive statements of the projects. 

A. Guidelines 

The project staff was concerned in meeting with people to discover 
information in these areas: 

- What motivated your use of technology? 

- Why did you choose this particular configuration? 

- Who made the decisions? 

- Who allocated the funds and/or determined cost effectiveness? 

- What predictors of success did you employ? 

- Hew did you define the problem and did you consider alternatives to 
this analysis? 

- Have you found your installation successful? Why? 

- What major problems have been encountered? 

- Could you have avoided these difficulties through improved • lanning? 

- What have been the reactions from students, teachers, and others? 

- In what way were these people involved in the planning? 

- Are their reactions justified? 

- What spin-off effects have you encountered which were not predictable, 
or which were predictable but were originally discounted? 

- What kinds of information would have helped you improve your planning? 

- Did you employ systems analysis techniques (including PERT, CPM, etc.)? 

- Did you employ a conscious process of change, innovation, and adoption? 

- What elements of your system (e.g., buildings, environments, student 
personal needs, etc.) did you plan for which later proved to be 
unnecessary parts of the plan? 



0 



83 



- What evaluation criteria did you pre-establish? How? Why? 

- What will be your basis for determining the success of the installation? 

- How do you see a simulation laboratory (or other prior-to-installation 
procedure) aiding your planning by providing inputs not presently 
available? 

In addition, information was requested on costs of the installation (per 
student or on some other basis) , the relation of these costs to planning 
procedures, the potential predicted for increased cost efficiences, and 
the effects of cost efficiencies on benefits lost or added. Some 
discussions were undertaken to determine the factors which determined the 
need for change and the resources considered critical. These discussions 
generally proved unsatisfying, since there appeared almost a universal 
lack of concern for considering (or for reorienting) extant patterns of 
thinking about resources. 

An attempt was made to visit a representative example, in an actual 
school setting, of each new technology operating. Meetings were also 
held with specialists in the design of component aspects of new systems, 
especially those aspects not usually connected with novel hardware or 
software, such as facilities, furnishings, and textbooks. The intention 
here was to determine the relevance of these components to new systems, 
and to determine to what degree multi-media technology was affecting the 
design of items in the educational environment not usually associated 
with technology. It was hoped that several examples of integrated 
technological planning could be observed in addition to the installations 
of specific technologies, but this proved more difficult than was antici- 
pated. Most of the apparently integrated operations proved to be based 
either on an electronic development to which appropriate teaching styles 
were added, on a clever arrangement of many individual devices (adding 
convenience but not originality) , or on the application of some advanced 
nuance of a standard psychology. In this respect the present state of 
educational technological practices remains disappointing. 

B . Institutions and Individuals Contacted or Visited 

The following institutions and/or individuals were visited or contacted 
during the course of the study for the purpose described at the right. 



Location and Primary Contact Activity, Additional Contacts 



Dr. Louis H. Brown 
Educational Media Specialist 
Institute 

University of Colorado 
Boulder, Colorado 



Discussion of factors inhibiting 
effective use of media in the schools 
research, development, and training 
needed for more effective use of 
media. 

Eugene Waldmann, 

Colorado Department of Education, 
and other institute members 



Location and Primary Contact (cont.) Activity, Additional Contacts 



Mr. Alan Green 
Educational Facilities 
Laboratories 
New York, New York 


Discussion of design factors in 
media planning, the RPI model 
classroom, and EFL as a type of 
laboratory. 


Mr. William Ryan 
Office of Institutional 
Research 

Rensselaer Polytechnic Institute 
Troy, New York 


Discussion and tour of multi-media 
installations including the model 
classroom. 

Dr. Walker, Professor of Biology 


State University of New York 
Albany, New York 


lour of campus, including multi- 
media, library, and CAI facilities. 


Dr. William McKeefry 
Academic Dean 

Southern Illinois University 
Carbondale, Illinois 


Small group conference on the 
design and use of the multi-media 
classroom and student response 
system. 

Mr. Lloyd Rivest, GE R&D 
Mr. Dick Rankin, GE R&D 
Mr. A. S. LeBlang, GE R&D 
Dr. Kenneth Fishell, Syracuse 
Uni vers tty 

Prof. Carrol Truss, University 
of Miami 


Dr. Donald Ely, Director 
Center for Instructional 
Communication 
Syracuse University 


Discussion of multi-media planning 
and research; tour of the multi- 
media, response classroom. 

Dr. Kenneth Fishell 
Mr. Michael Molenda 
Dr. DeLayne Hudspeth 


Mr. Charles Sheridan, Manager 
SDC, Rome Facility 
Rome, New York 


Problems of design and implementation 
of CAI and computer-based instruc- 
tional programs for neighboring 
secondary schools. 


Dr. Patrick Suppes 
Mr. Cornelius Butler 
Ventura Hall 
Stanford University 
Palo Alto, California 


1. IBM 1500 Computer Tutorial mode, 
Brentwood School, Ravenswood 
School District, East Palo Alto, 
California 

Mr. Karl Anselm, Director 
Mr. Bill Rybynsky, Principal 

2. Drill-and-Practice CIA Proprams 

Dr. Max Jerman, Director 
Miss Luanne Berkowitz, 
Curriculum Specialist 
Clifford Street School 
Redwood City, California 
Mr. Frank Messer, Principal 



Location and Primary Contact (cont.) Activity, Additional Contacts 



Dr. Donald Tosti 
Westinghouse Learning 

Corporation Research Center 
1840 Lomas Street 
Albuquerque, New Mexico 



Station KNME (ETV) 

Dr. Claude Hempen 
Station Manager 
1801 Roma Street, N.E. 
Albuquerque, New Mexico 

Dr. Carol Nolan 
CCTV Consultant 
Chicago Public Schools 
228 N. LaSalle Street 
Chicago, Illinois 

Evanston High School 
School 

Dr. Lloyd S. Michael, 
Principal 
1600 Dodge Avenue 
Evanston, Illinois 

Evanston Elementary 
District //65 
Dr. Gregory Coffin, 
Superintendent 

The Raymond School 
Dr. Raymond Jerrems, 
Principal 
37 Wabash Street 
Chicago, Illinois 

SDC Research and Technology 
Division 

Santa Monica, California 



Media Teaching Project (AACTE) 
Miss Freda Douglas 
1246 16th Street, N.W. 
Washington, D. C. 



1. Contingency Management Media 
Projects 

Mr. Ken Kammerman 
Dr. Lloyd Homme 

2. Educational Achievement Center 

Mr. Clifford Chadwick 
Mrs . Barbara Salazar 

ETV Studio and Personnel 
Dr. W. Bundy 



The Chicago Cluster CCTV Project 
The Byrd School 
Dr. Hogan, Principal 



Integrated multi-media high school 
program, including CCTV, VTR, Dial 
Access (proposed) , and classroom 
media. 

Mr. Phil Frost 
Mrs. Wanda Mitchell 

Proposed laboratory school plans 
and board of education committee 
to study technology. 

Dr. J. Robert Parkinson 

Classroom media and non-media 
oriented instruction in an inner- 
city school. 



Seminar on the Feasibility of Multi- 
Media Simulations and Model Designs 
.Robert Filep 
Gloria Grace 
John Coulson 
Harry Silberman 
Gerald Newmark 

Use of Media and Teacher Instruction 
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Location and Primary Contact (cont.) Activity, Additional Contacts 



National Association of Educa- 
tional Broadcasters 
1346 Connecticut Avenue, N.W. 
Washington, D. C. 


Colloquy on Models and Strategies 
for Media Planning, Implementation, 
and Systems Design 
Mr. Lewis Rhodes 
Mr. James Fellows 
Mr. A1 Friedman 


U.S. Air Force Academy 
Ma j . Michael J. Grady Jr. 
Directorate of Instructional 
Technology 


Visit facilities and review program 


Oral Roberts University 
Tulsa, Oklahoma 


Learning Resources Center 
Dial Access System 


Dr. Ralph Gerard, Dean 
University of California, 
Irvine 


CAI and Computer facility 


Project GENESYS 
University of Florida 
Cape Kennedy, Florida 


Closed circuit television 


Melbourne High School 
Melbourne, Florida 


Multi-media Techniques 
Individualized Instruction 


Oakleaf School and the 
University of Pittsburgh 
Learning Research and 
Development Center 


Instructional management system 


Dr. Ray Cobb, Director of 
Education 

Government of American 
Samoa 

Pago Pago, Samoa 


Television-assisted instruction 



C. Comments on Representative Visits 



Oral Roberts University. Tulsa, Oklahoma 

The dial access component of the Learning Resources Center at Oral 
Roberts represents perhaps as sophisticated a technological implementa- 
tion for education as exists anywhere. Twenty-five audio channels and 
three video channels are available to over 160 student carrels located 
throughout the library, in classrooms, and at terminals in dormitories. 
The system appears to be in relatively constant use. In addition to 
this operation, classrooms are equipped with specially constructed 
teachers’ desks containing tape recorders, slide consoles, and over- 
head projectors. Many classrooms are equipped for television (CCTV, 

VTR, and open reception). There are several front projection multi- 
media classrooms with response apparatus. Back-up facilities include 
a fully equipped television studio, graphic arts rooms, and a complete 
media library. There is no question that this building represents as 
complete a unification of teaching resources as is presently available. 

One difficulty with the planning of the Learning Resources Center was 
pointed out rather strongly. There was little involvement of the 
faculty in the initial planning and installation of the equipment. 

While some resentment has resulted, the more serious effect is a 
comparative lack of creative implementation and use of the resources for 
affecting changes in the teacher-learning process. For the most part 
the tapes available on dial access are old lectures given by professors. 
Some use is made of the graphic facilities for slides and overhead, but 
apparently there has been little real change in the teaching process. 

The director of the learning resources center indicated that beginning 
this next year, all faculty will be encouraged to submit proposals for 
the redesign of their courses following the principles of a learning 
system. Direct faculty instruction in systems planning techniques 
will precede these proposals. There is reflected in this action a 
genuine concern that the facilities could be used in a more appropriate 
or cost-effective manner. 

University of California. Irvine 

It is the intention of the University of California, Irvine, to imple- 
ment a "computer facility" on the campus so that virtually all adminis- 
trative, research, and instructional segments will be controlled by a 
system of computers. The expectation is for over 700 terminals to 
handle the teaching load for the entire student body. Irvine clearly 
considers itself, and is acknowledged as, a leader in the development 
of computer applications in education. 

At this juncture, there are twenty-four operating terminals servicing 
approximately one-third of the student body in various ways. Some use 
the computers only for computation; others for review (drill and 
practice). Only one course, Sociology I, is completely contained with- 
in the computer. Students interact with a terminal, receive reading 
and other assignments from the computer, and do not have contact with 
a "live" professor during the course. Although one may express some 
concern that the initial use of full computerization occurred in this 




88 



critical human interaction area, it is still too early to judge the 
success of the involvement with the machine. 

There are a number of problems at Irvine. Some of these arise from 
the nature and the condition of the computer language being developed 
and used. Some aspects of the language have not been finished, and 
other problems are being dealt with in the initial runs. The scarcity 
of computer programmed material limits the forward movement of the 
operation. Irvine has chosen not to share programs from other campuses, 
one important source of additional material. A critical problem is 
money . 

The Irvine campus is on the road toward developing a powerful and 
sophisticated technological system. Yet it is not possible to escape 
serious doubts about the pedagogical implications of these applications 
unless greater change is made in teacher-learner interactions, the 
nature of the learning process, the scope of materials and experiences 
presented to the student, or the nature of higher education. This 
last point is critical: advanced technologies can and should lead to 

a reorganization and revitalization of the university function; they 
should not be merely sophisticated devices for storing and presenting 
acknowledged ineffective systems of education. There is too little 
evidence, however, that such a reorganization of learning is given 
high priority at present. Instead, immediate pressures emphasize getting 
new technological machinery into smoothly running condition, without 
great regard for interrelations with the overall educational system. 

U. S. Air Force Academy 

The Directorate of Instructional Technology at the United States Air 
Force Academy, Colorado Springs appears to represent an ideal instruc- 
tional technology component within a total educational program and 
agency. Their operating budget is large, they are thoroughly equipped 
and adequately staffed, and they provide broad services tc all parts 
of the Academy within an appropriate organizational context of 
responsibility and interaction. Essential features of the program 
appear to be the splendid equipment and support situation and the fact 
that individual users are not billed for services. This situation 
encourages experimentation and continued use, since users can try any 
mediated technique without direct effects on their own departmental 
budgets. The Academy is presently constructing a total of 168 class- 
rooms, arranged in clusters of five, for the use of CCTV. In addition 
several multi-media classrooms are under construction, designed in 
part after one in use at the Harvard Business School. These rooms will 
provide for the use of CCTV as well as front screen and rear screen 
projection. A responder system will be available. 

Project GENESYS and Melbourne High School 

Contrasting uses of media can be seen at the University of Florida’s 
Project GENESYS installation at Cape Kennedy and at the nearby Mel- 
bourne High School, Melbourne, Florida. Project GENESYS results from 
a deliberate plan for the use of CCTV to solve the problems of space, 
time, and communication which arise in a program of advanced, specializ- 
ed education for scattered, part-time students. Each of the three 
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off-campus GENESYS stations provides for on-site professional staff, 
some research facilities (mostly staff-oriented), a library, and live 
lecture situations, since each station broadcasts to the others. The 
CCTV is equipped with a telephone feedback, so that a student may query 
a professor at any time. The studio is equipped with a teacher-operated 
zoom camera and a closeup camera for transmission of printed matter. 

The system employs high resolution receivers. During broadcasts a 
teacher may operate his own cameras, but a technician is on hand at all 
times. Several of the reception rooms are small and allow for group 
discussion during broadcasts, although attention to the lectures would 
tend to render this situation inappropriate. Nevertheless, this 
appears to be a unique opportunity for student-to-student interaction 
during the learning situation. Although a feedback method may not 
always be necessary, its use in this project is important. Generally, 
the project's design quality and appropriateness in terms of stated 
problems are impressive. 

Melbourne High School uses only the minimum of media, but each use 
appears excellent. A large group in art is instructed through the use 
of an overhead projector, the teacher creating the transparencies a- 
head of class. Approximately three hundred students receive instruc- 
tion in typing simultaneously through the use of a loudspeaker broad- 
cast system in an oversized classroom. One double class — roughly 50 
students — in a slow learning core situation (history and English) con- 
tains virtually all of the usual classroom media devices, including a 
transparency-making rig. The head teacher is supported by an art 
teacher whose special interest is in the use of visuals in education. 
Together they are designing some interesting programs for this group. 

The major proposed additional use of media in this high school will be 
for increased individual instruction. The tutorial program at the 
school requires that attention be devoted to providing more individual 
experiences and a greater menu of instructional resources for the 
broad spectrum of projects undertaken by students. Media are seen as 
necessary parts of this program, although planned costs may not be 
justifiable except on an experimental basis. Melbourne is an excellent 
high school, thoroughly designed in its various aspects. Within this 
total design, media can play a part; but unless media are carefully 
implemented, their use could throw the school's overall program out of 
focus. 

The Oakleaf School, and University of Pittsburgh. Learning Research and 
Development Center 

Members of the project staff visited the Oakleaf School and representa- 
tives of the Learning Research and Development Center, University of 
Pittsburgh, to explore the development of Individually Prescribed 
Instruction (IPI) . This experience proved to be highly provocative and 
illuminating in examining the use of individual instruction and the 
selection of media appropriate to the nature of the learning require- 
ments. The system appeared to be highly effective for improving learn- 
ing in the target subject areas, although adequate evaluative data were 
not available. Discussion with the principal, Dr. Donald Deep, revealed 
a high level of satisfaction with IPI but indicated some significant 
omissions in the total plan. 
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At this stage, the Oakleaf Project has apparently not addressed itself 
to the statement of long-range goals or the purposes of education for 
the achievement of these goals. M What do you want the product (i.e., 
the student) to look like when he leaves, and how does IPI contribute 
to this aim?" was not a first question in the planning or a comfortable 
one now. In addition, while the IPI portions of the day were carefully 
designed and controlled, the rest of the school day was not so designed 
or monitored, so that values other than those implied by the IPI 
experiences are still subject to chance occurrence. 

Brentwood School (Stanford University) 

The Stanford CAI system is designed to present instructional materials 
to 16 students simultaneously, permitting each student to work at his 
own pace and on different learning sequences. When an instructional 
sequence is completed, a complete record of the sequence of materials 
presented to the child and a chronology of his responses is made 
available. 

The heart of instruction for the student is the rewarding, sensitive 
voice in the headset. This computer-operated teacher responds through 
circuitry at the speed of light to the student’s manipulation of the 
lightpen and typewriter input devices. The teacher's voice directs 
the student to look at an image on the TV screen. For example, a 
first grader might be asked to ncte that a horse and a cow are at 
either ends of the screen, near the top. At the bottom of the screen 
is a farmer. The child is then asked to touch his light pen to the 
TV screen and to move it along the screen to show how the farmer could 
get to the horse. (The computer causes a flashing arrow to appear in 
front of each of the described objects, horse, cow, farmer, as each 
word is first repeated) . The computer then assesses whether the child 
responds acceptably and at what degree of acceptability • If the child 
responds within a reference frame acceptable to the curriculum designer, 
the story or problem continues. If not, the problem can be presented 
in another manner. The computer can distinguish the student who 
doesn’t know the difference between a horse and a cow. In such case, 
the computer activates the random access projector and a full color 
illustrates and narrates characteristics of various animals. The 
sequence is as available to new direction as there are ideas in the 
minds of curriculum workers. 

The student is able to interact kinesthetically with a computer program 
by touching his light pen to a TV screen (a cathode ray tube) . The 
student also reacts to visual clues on another display screen* follows 
his teacher’s voice in the headset, uses a typewriter input device, and 
tapes his own verbal responses. If one accepts as a learning premise 
that the greater the number of senses brought into the teaching 
situation, the greater will be the degree of understanding, then CAI 
of this degree of complexity should be superior to CAI with typewriter 

interaction only. 
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The Byrd School, Chicago « Illinois 



A television studio, including control room VTR remote facilities, 
auxiliary offices, and materials preparation facilities, is located in 
the Byrd Elementary School. By means of rented lines, programs of this 
studio’s two channels are transmitted to four other elementary schools. 
These five schools form a "CCTV cluster", and there is some established 
interaction among them as a result of the TV project. 

Each classroom is equipped with two television receivers capable of 
receiving regular broadcasts and (through signal translation) the ETV 
broadcasts, as well as the two CCTV channels. Advanced notice is 
provided the classroom teacher of special materials required to 
accompany broadcasts but no particular readjustments were noted. 

The television team consists of a two-man technical staff, one full- 
time television teacher (a specialist in elementary science) , and a TV 
coordinator. This individual works with teachers, develops methodo- 
logies for classroom implementation, assures that supplies are available, 
and provides feedback and assistance with problems. 

A brief description of an observed television lesson follows: Two 

television receivers were at either side in front of the students. A 
globe was on a table, centered between the television sets, because 
the program was concerned with how to use a globe. The students had 
evidently cleared their desks in preparation for the broadcast. The 
television teacher asked questions and suggested activities to involve 
the students, but there seemed a distinct unwillingness to respond. 

Since they did not have paper or pencils available, they were unable 
to jot downs words or ideas from the broadcast. The classroom teacher 
circulated (patrolled) the room, occasionally admonishing students. 

The television teacher talked too fast and many of her map and word 
displays were unclear. There seemed, in short, little appropriate 
learning design or use of the media. As an isolated case, this observa- 
tion is of little significance. Unfortunately, however, too many 
instances of the use of educational television illustrate similar 
deficiencies. 

The Samoan Educational System 



The educational system of American Samoa has undergone almost a complete 
overhauling since 1961. A significant element in this change has been 
the design and use of an educational television component. The effect 
has been a virtual revolution in educational thinking and planning, 
especially in the deliberate application of an advanced technology 
to the solution of persistent and chronic educational problems. Since 
a tremendous amount of information has been published on the Samoan 
experiment, this report will be limited to a short section of background 
information and a description of aspects of particular interest to the 
present project. 

American Samoa consists of seven islands located in the South Pacific, 
with an estimated total population of about 25,000. The largest island 
of the territory, Tutuila, is the home of about 80% of the population. 
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In 1961, only about one-half of the eligible children were attending 
school with any regularity. The schools they attended were grass-roofed 
huts, typically poorly maintained and ill-equipped. Their teachers had 
the equivalent of an elementary school education; none possessed main- 
land teaching certification; and most of them could barely speak English — 
although all instruction was supposed to be in the English language. 

Since then, the educational system has undergone a revolutionary process 
of change. In all-new consolidated schools the Samoan child views tele- 
vision for from one-quarter of the school day in the early grades up to 
one-half of the school day in the high schools. Each classroom also has 
a native Samoan teacher who views the. television lessons along with the 
students, manages the operation of the classroom, and attends to indivi- 
dual needs in preparing for and following up the television lessons. 

All of the television programs and printed materials used in the schools 
are prepared at the Department of Education's Educational Resources 
Center especially for use in Samoa. 

The television medium has an enormous potential for bringing rich and 
varied educational experiences to its receivers. Most importantly, it 
has the potential for being immediately responsive to the changing needs 
of its users. New content or teaching methods can be diffused through- 
out the system immediately, without having to wait for the training of 
a new generation of teachers or the retraining of the present ones. 

Lessons can be repeated, altered, or delayed upon demand. Similarly, 
the curriculum of the schools must constantly be revised to keep pace 
with the accelerating changes in the Samoan culture, the growing linguis- 
tic and mental abilities of students at all levels, the rapidly changing 
demands of the world of work, and the new interests of children experi- 
encing a society in transition. 

There are four essential elements in the present educational system: 
the production process, the television transmission system, the lesson 
design, and the local school. The design of each element seems to have 
hinged on first a careful analysis of the nature of the Samoan child and 
the particular educational problems resulting from peculiarities of 
language and culture, and second on a careful analysis of the problem. 0 
imposed by the nature of the extant system, especially the critical 
problem of the shortage of qualified teachers. 

When planning for the new system began in 1961, the primary instructional 
mode was rote learning of verbalized English. Neither the teacher nor 
the student, however, understood the English, and so the repetition was 
meaningless. As well, the teachers were highly authoritarian; questions 
or indications of a failure to understand were not permissable student 
responses. The first concern, therefore, was to insure true teacher- 
student communication in the learning process through providing meaning- 
ful learning situations and training the teachers in new modes of 
behavior. 

Following the decision to use as much English as possible, but not to 
demean the Saiuoan dialect as had previously been done, Dr. George Pittman 
directed a thorough analysis of the native speech and its relation to 
English. This analysis made it clear that commercially produced texts 
were totally useless, and the decision was reached to produce all 
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materials locally. In advance of a lesson, usually a week, all teachers 
receive study guides directing their activities during the pre and post 
television sessions. These guides include equipment lists, motivation 
suggestions, actual questions, and statements of learning goals for the 
lesson. Teachers prepare themselves in accordance with these guides, 
and they meet with the mainland principal for inservice education based 
on them. The stronger teachers tend to go beyond the guide, but the 
unsure individual is well supported by its careful delineation of 
functions and its suggestions for questions. Many learning materials 
have been produced in addition to the lesson guides. These include 
some very handsome readers both in English and Samoan. 

Almost all lessons are videotaped in advance, partly because there are 
only four studios to supply six channels, but also because the television 
teachers are then free to watch the broadcast from a school room. Since 
the broadcast schedule dictates that the television lesson must be seen 
at a specified time, all other activities in the classroom revolve 
around this factor. All students in a given level are, therefore, 
studying the same material at the same time. The broadcasts, however, 
represent only 25% of the school day, so the feeling of regimentation 
is avoided while, on the other hand, it provides a firm structure 
for the Samoan teacher. 

The classrooms in the consolidated schools were designed for television 
reception. A 21" receiver is directly in front of the class at an 
appropriate height. Just before the lesson a student opens the closet 
doors to expose the TV, turns it on, and shuts off the lights. The 
picture and the sound are clearly seen and heard at all points of the 
room. After the lesson, the student shuts the closet, and the room then 
appears as any non- television classroom. 

The children greet the TV teacher, respond as directed, and are 
genuinely involved in the lesson. This is not passive viewing of 
entertainment or "enrichment" • The TV is an active (and interactive) 
part of the child’s learning process. It is this quality of the tele- 
vision experience that is most difficult to explain and will remain 
only vaguely appreciated unless observed. This aspect of rapport with 
television is perhaps at the crux of the technology in education issue, 
since many critics point to the dehumanizing characteristics of machines. 
In this case, television functions in appropriate ways to perform only 
part of the learning task, but in addition the children respond to two 
aspects of the broadcast. First, the television teacher talks directly 
to the child, says good morning, etc., and they respond by name. This 
and similar devices insure a close rapport and, in fact, develop a 
warm feeling in the children for their TV teachers. Second, the 
children respond to the quality of the instruction and the quality of 
the lesson. Attention to the television teacher is greater than it 
would be for the classroom teacher because the presentation is superior 
for those parts of the total instructional process found appropriate 
for television. There is no question about the appreciation for quality 
in the instructional process on the part of even the youngest children. 
The television teachers also visit classrooms in person, talk with 
students as well as teachers, and receive daily feedback on the quality 
of their performances. These measures certainly add to the interpersonal 



nature of the presentation, but are secondary still to the characteris- 
tics of appropriateness and quality. 

The lesson format seemed to be the same for all the classes observed, 
and quality did seem to emphasize the aspect of regimentation. The 
format is extremely effective device for insuring the adequate develop- 
ment of instructional presentations. There did appear to be some un- 
necessary duplication between the local teacher’s activities and the 
television broadcast, but this could be the fault of the individual 
teacher rather than of the plan. There also seemed less than sufficient 
planning for individual responses (usually students responded in a group), 
a lack of communication between and among students (the primary pattern 
remains teacher-to-student : ask; student-to- teacher : answer), and very 
little imaginative or creative interpretation of the lesson. 

Critics of the Samoan experiment have noted a lack of empirical data on 
achievement and on the operation of the system. There are, nevertheless, 
several favorable elements obviously present. First, there is a well 
run, creative atmosphere caused by a dedicated, sincere, and talented 
staff. Second, there is a strong sense of purpose and contribution on 
the part of the staff, and there is excitement, attention, and involve- 
ment on the part of students. Third, there are still signs among the 
faculty and in the schools of changes so radical that adjustment to the 
upgrading and improvements is a continuing activity. 

The Samoan system spends under $500 per child per year (not including 
the initial capital investments) . Within this budget all curricular 
materials are planned, developed, and written; the television system is 
supported; and transportation, food, and support services are funded. 

This cost figure is not directly comparable with the mainland because 
local teachers receive considerably smaller salaries than their 
equivalents in the states. Their considerably inferior training and 
somewhat less total sphere of responsibilities, however, forces the 
question of appropriate staffing (i.e., increased use of paraprofessionals 
in support of mediated efforts) and the concomitant reorganization of 
critical resources within a school district. 

D. Summary of Visits and Interviews 

The general impression gained from these experiences might be stated 
briefly in this way: t here is a general lack of systematic and thorough 

planning in the application of multi-media to perceived problems of 
educational improvement . Having made this statement, it is immediately 
necessary to qualify it by indicating that many projects are motivated 
by the need to develop and test a hardware or software system, and that 
their justification is to be found within an experimental or developmental 
context. Usually, however, results are not applicable to another situa- 
tion, nor is information collected to permit the comparison among 
alternatives. 

As stated throughout this report, the chief cause for the problems in 
present media use appears to stem from either a lack of perception or 
a lack of appreciation for the necessity to redesign the total learning 
situation, or at least to begin with a clear statement of learning goals 
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and a comprehension of the system presently working to achieve these 
goals. Media can perform certain functions which are appropriate to 
their nature and which can support learning. They cannot, however, 
replace or supplant a teacher performing these same functions unless 
the role of each is designed in relationship to the other. The necessity 
for designing man-machine systems and for analyzing critical needs which 
then can be met by alternative methods cannot be overstated. 

Media planning should begin with a problem, not with a technique. Such 
problems have included individualizing instruction, improving intra- 
school communication, overcoming a shortage of expert teachers, trans- 
mitting at great distances, or equalizing educational opportunities for 
deviant populations. The contributions of various media to such 
problems can then be judged in relation to the othar factors in the 
situation, alternative means, and the values of the community. Far too 
often, however, commitments have been to techniques rather than to the 
development of optimum solutions to problems. As has been stressed 
throughout this report, the effective use of media in such situations 
may require designing beyond the customary limits of institutional 
thinking. For instance, there is little realistic value in an 
expensive dial-access system to store a professor’s lectures if the 
lectures are either poor or dated, and if their replay results in only 
test passing rather than interactive learning (unless the values for 
the system are consonant with its use). 

The greatest difficulty with present experimentation and implementation 
activities with media stems from the fact that they are generally 
promoted in a context of protective enthusiasm. This results in low 
motivation for problem analysis, clarification of objectives, design of 
alternatives, cost analysis, systematic information collection, and 
objective evaluation. Each situation considers itself unique, probably 
a necessary concomitant to the obtaining of support, but this straining 
for individuality precludes objectivity and replication. One frequent 
explanation for a reluctance to be concerned with costs is related to 
the relatively high costs attached to experimentation and development. 
Most innovators feel their project includes expenses which would not be 
involved in future installations. In some cases, this seems true, but 
it does not detract from the need for careful accounting during the 
decision process and installation. 

In most media implementations, initial decisions are made by the formal 
leadership, with little involvement of faculty or students. This 
results in later difficulties in the acceptance of the innovation. In 
cases where the faculty, or at least their representatives, had been 
involved, there was less tension and greater acceptance. 

Despite these criticisms of the planning process, some excellent 
installations of particular media were observed. Although each evidences 
problems, the direction of the future is clearly toward breaking away 
from past media limitations. 
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Appendix C. Some Previous Experiments with Simulated Environments 



The use of simulated environments for the study of educational pro- 
blems has not been undertaken on a large scale, although two signifi- 
cant developments were reported in the body of this document. There 
exists, however, a substantial body of experience with the applica- 
tion of simulation techniques to other areas. These simulations have 
on occasion been limited to physical representations, but they have 
also included the development of complex computer programs for simu- 
lated training and longitudinal research. This appendix will present 
a brief review of some major simulation efforts within SDC. Some of 
these projects offer insights concerning the potential of simulation 
techniques for the field of education. 

A separate project, currently in its final stages, has involved a 
study of past simulations and related experiments. The work, directed 
by Henry M. Parsons, is entitled A Review and Analysis of Man-Machine 
System Exper iments and Laboratories , sponsored by the Aerospace Medical 
Research Laboratories. Parsons has cataloged virtually every major 
simulation laboratory and has reviewed the strategy and methodology 
of man-machine experimentation. His work has provided an extremely 
important input to the present study. 

Many of the simulation projects described below embody techniques of 
doubtful applicability to educational problems. It would be a fatal 
error to establish an "educational simulation laboratory" on the basis 
of a primary concern with simulation techniques. Educational problems 
must determine which techniques are to be used to solve them, rather 
than letting techniques determine whether and how educational problems 
are to be approached. The main text above discusses uses of simula- 
tion for multi-media problems. This appendix is provided for the read- 
er who wishes further concrete examples of the uses of simulation 
methods . 

Emergency Operations Simulation Research Center 

System Development Corporation maintains a facility in Santa Monica 
capable of simulating environmental emergencies and hazards. Called 
the Emergency Operations Research Center (EORC) , the center offers 
the ability to evaluate and realistically practice methods of coping 
with emergencies without waiting for a real crisis to prove or dis- 
prove their effectiveness . The EORC makes it possible to examine 
and strengthen an organization's command relationships, communications, 
decision-making effectiveness, resource distribution and use of informa- 
tion under stress. 

The EORC is capable of simulating air pollution episodes, such as 
occurred in Donor a, London, New York City and elsewhere, and the re 
sponse of official agencies and the public to these episodes. It is 
capable of simulating a second and third stage Los Angeles alert and 
the effectiveness of plans to curtail traffic and to shut down industry. 
This process can involve responsible officials at all levels of govern 
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ment and the experience can contribute to achieving cooperation and 
management breakthroughs in the solution of chronic problems. It can 
also contribute to a realistic analysis of the entire air pollution 
management system. 

Constructed as a general-purpose laboratory, the Emergency Operations 
Research Center was designed to meet a wide variety of experimental 
situations within a single facility. Most walls and fixtures are 
easily movable, thus allowing for rapid and economical changes in con- 
figuration. The laboratory is divided into two major areas (see 
Figure 3). The first contains facilities for briefings and the actual 
conduct of simulation. The second is designed for use by participants 
in either single or multiple experiments with adjacent observation 
decks. Therefore, the design and physical layout of the EORC effective- 
ly provides facilities for all aspects of operations exercises; brief- 
ings; recording, monitoring and analysis; realism in the simulation; 
and debriefings. 




1 Simulation Laboratory Foyer 
4 Simulation Area 
7 Emergency Operating Center Area 



2 Briefing Area 
5 Communication Center Area 
8 Observation Deck ( movable ) 



3 Observation Area (raised platform) 
6 Observation Deck (raised platform) 
9 Small Experiment Area 



Figure 3. Emergency Operations Research Center 






Cameras in the EORC provide SDC analysts with information on partici- 
pants' actions. Wired into the laboratory's own telephone switchboard 
are monitor cabinets allowing observers to listen in on phone lines 
assigned to them. Tape recorders are also connected to the phone lines 
to operate whenever the phones are used. A recorder started with the 
exercise gives the experimenters a time reference as a base for reduc- 
ing and analyzing the data. Teletype input and receiving equipment, 
closed-circuit television, a 14-channel simulated radio system and a 
public address system allow observers and subordinate, lateral and 
higher echelons of authority to present messages to the group being 

exercised. 

Observation decks designed specifically for this facility allow critics 
and students to note the effects of actions taken in the control of the 
simulated disaster. To prevent the presence of observers in the deck 
area from distracting the exercise participants in any way, the deck 
is provided with one-way glass and is completely soundproofed. 

During the exercise, the participants represent the major service 
organizations of an actual city. Thus, the knowledge they gain during 
the exercise can help them improve real emergency operation plans. 
Personnel in the simulation area adjoining the operations area note 
the effect of participants' actions and also present new situations 
to the operating staff. These same people often assume the roles of 
the lateral, subordinate and higher echelons of command. 

Digital Flight Simulator Study for the C-141A Jet Tra nsport 

SDC conducted a study of two different types of digital flight simula- 
tors for the C-141A jet transport, its purpose being to analyze, compare, 
test, and evaluate the twc simulators from an overall system performance 
viewpoint. One of these simulators is activated by two medium sized 
general-purpose digital computers. The other simulator is activated 
by a special-purpose drum- instruct ion memory computer. The comparison 
includes an analysis and discussion of: (1) equipment characteristics, 

(2) mathematical modeling techniques, (3) software capability, 

(4) adequacy of system documentation provided, (5) simulator usage, 
and (6) acceptance testing. 
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ERIC 



Simulated Naval Plot (SIMNAVPLOT) 



SIMNAVPLOT was developed by SPC as part of a research project involving 
a laboratory simulation of a Naval command post in operation, support- 
ed by a data base system and displays. The operational mission of the 
command post was the surveillance of ocean traffic, the classification 
of situations at sea that might require the alteration of tactical 
Naval forces, and the allocation of forces to respond to these situa- 
tions. 

SIMNAVPLOT was comprised of a simulated command-post staff and a com- 
puter program system operating under the AN/FSQ-32 time-sharing 
system to handle the simulated functions of surveillance, classifica- 
tion, of situations, force status monitoring, and force allocation. 

The primary purpose of the data base and simulation development was 
to test the effect of variations in equipment configurations, staff 
composition and organization, and operting procedures on command 
post performance. 

Digital Simulation for the Orbiting Astronomical Observatory (0A0 ) 

In support of NASA's scientific space satellite program, SDC performed 
as technical advisor and as developer of software for ground-based 
computer control functions. In initial work, SDC performed an 
analytical research study to determine the computing system required 
in support of the Orbiting Astronomical Observatory. 

A major product of the project was a Digital Acceptance Test System 
(DATS), which was composed of an acceptance test plan, acceptance test 
specifications, and a simulator of the spacecraft and control station 
system. DATS testing was extended to the OAO's Operational Control 
Computing System (OCCS), the system that controlled spacecraft opera- 
tions in orbit. The DATS provided digital simulation of all OCCS 
functions, including the transforming of experimental requirements 
(list of stars to be observed, etc.) into commands to the spacecraft, 
reading out status and experimental data, pointing the spacecraft in 
the proper direction, and monitoring spacecraft status. 

Model for Scheduling Satellite Tracking Operations 

This model is designed to schedule the major equipments and operations 
of a satellite tracking network by means of computer simulation. 

Inputs to the model define such factors as the scheduling period, 
stations, satellites, priorities, and equipment usage. The model 
performs many functions that make it valuable as a planning tool. 

Among these are the following: 

- Computes the rise and set times for every pass that each vehicle 
makes over every station. 

- Detects conflicts in which: 

(a) two or more vehicles art over the same station at the same time, 
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(b) two or more stations would see a vehicle at the same time, 

(c) the radio frequencies of two or more vehicles would interface. 

- Resolves the conflicts among vehicles and stations by means of a 
variety of priority schemes, dependent upon the values of certain 
input parameters. 

- Produces summary and statistical data on utilization of system 
equipment . 

- Schedules for as many as thirty-six vehicles and thirty-six stations 
in operation at the same time. 

- Permits selective suppression of outputs. 

- Provides for orbital parameters to be input by various modes to 
suit the user. 

Airborne Warning and Control System (AWACS) Simulation Vehicle 

A ground-based or airborne command and control system is simulated 
for the control of friendly fighter aircraft. This system can main- 
tain radar surveillance of the airspace, and direct friendly aircraft 
in accordance with pre-defined algorithms. 

Friendly fighter aircraft on ground alert may be simulated and assigned 
to hostiles that are being tracked. After assignment, a fighter is 
scrambled and is directed to intercept the hostile aircraft. Upon 
reaching the vicinity of the hostile, a combat phase maneuver is 
simulated. At the end of the mission, the fighter is either placed 
on Combat Air Patrol or returns to the nearest base, depending on its 
weapons and fuel status. 

Operator intervention is provided. The AWAC aircraft flight path and 
the routes and assignments of the fighters can be controlled to 
simulate differing operational concepts and tactics. 

Some events that occur in an air battle are predictable only on a 
statistical basis, and a Monte Carlo routine handles such events. As 
an example, friendly airborne aircraft are subject to abort throughout 
their flight on a random basis, at some prescribed rate, requiring the 
control system to reassess the tactical situation and perhaps make a 
new assignment. 

Statistical analysis techniques have been devised to allow a maximum 
of information to be extracted from each simulation exercise or series 
of exercises. Using such techniques, statistical confidence factors 
can be assigned to the validity of hypotheses derived from the simula- 
tion results, and expected-value relationships can be derived from a 
limited number of exercises. 

The simulation vehicle has been used as a took in refinement of the 
Operational Employment Concept for AWACS in continental air defense. 



The sensitivity of system effectiveness to tactics and to hardware 
parameters has also been examined. 

A Dynamic Computer Model for Simulating Military Command Systems 

This model simulates a command system comprised of a command post and 
a network of subordinate weapon control centers, weapon launch plat- 
forms, weapons, sensor, and their interconnecting communication links. 

Its major purpose is to serve as a general simulation tool that can 
readily be adapted to simulate a variety of command systems and con- 
flict situations. As such, it can aid in evaluation of performance 
and effectiveness of command- control systems, operating as vulnerable 
networks in dynamic conflict with a reactive enemy. 

The model’s programs are highly parameterized, so they can be 
adapted conveniently to simulate many different types of military 
systems and conflict situations. It operates in either of two basic 
modes: (1) as an algorithmic model in which its programs run as a 

closed system within the computer; and (2) as an open simulation, in 
which humans in the command post interact "on-line" with the program 
system. The humans monitor the simulated conflict situations, as 
reported by the modeled sensors and weapon control centers, make 
special requests for information, and issue orders to commit weapons 
or to change their level of readiness. 

The approach used in design of computer programs follows closely the 
philosophy set forth by several simulation systems such as SIMPAC 
(SDC) , SIMSCRIPT (RAND) , and the General-Purpose Simulation System 
(IBM). The functional design for the model was first interpreted in 
terms of four basic simulation concepts: (1) Activity, (2) Operational 

Resource, (3) Queue, and (4) Message. 

An Activity is a basic information processing task in a flow network; 
it is distinct from the Operational Resource that is utilized by the 
Activity. In this model, an Activity is a program under direct 
control of the cycling routine; examples are the Engaged Sensor Activity, 
the Launch Control Center Activity, and Weapon Flight Monitor Activity 
(all in the Engagement Analyzer). The cycling routine is responsible 
for sequencing operation of Activities in time and for monitoring the 
model clock. 

An Operational Resource is the simulated man and/or machine that 
actually enables performance of an Activity. The Operational Resources 
are distinct from the Activity itself, and are maintained in the system 
data base. Examples of Operational Resources are sensors, weapons, 
and communication links. 

A Queue is a waiting line in which information to be processed is 
kept, and in which information already processed is placed. Most 
Activities have both input queues and output queues. Paths indicating 
flow of information through the system have been defined, when all 
Activities and Queues in the system to be modeled have been described. 
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The last modeling concept is that of Messages that flow through the 
system. Messages are generated by Activities, transferred between 
Activities through Queues, and then received and operated upon by 
Activities. Examples of types of messages in this model are the de- 
tection reports generated by sensor activities and orders to launch 
weapons generated by the Scenario (or humans) . 

The control program contains the Event Cycler, which controls the 
sequence for operating the functional modules and the input-output 
programs. During a simulation run, the Event Cycler checks each 
functional module and input-output program, in turn, to see if it is 
ready to operate. If, as a result of this check, the operation of a 
particiular functional module is required, the Event Cycler allows 
the module to initiate and complete the necessary computations, and 
then it resumes control. For example, if an order to fire a weapon 
has just been received by the weapon platform module, it will be 
ready to operate (that is, "fire" the weapon) when it is next checked 
by the Event Cycler. If the module is not ready to operate (for 
example, no "firing" orders received), it is bypassed and checked 
again during the next "cycle". The Event Cycler operates some of the 
functional modules on a periodic basis, that is, every A t minutes, 
where A t is a parameter. An example is the functional module that 
"moves" the weapons through space. 

All messages that flow between functional modules in the system are 
sent through the communications module, where appropriate delays are 
introduced. The communications module stores messages in temporary 
buffer storage, and sends them on after the appropriate time delay 
has passed, but only if the communication link is operating. 

Simulation of a Time-Sharing Computer System 

After the initial SDC Time-Sharing System became operational, a 
simulation model was constructed to predict and evaluate effects of 
proposed system design and hardware configuration modifications. As 
an evaluation tool, the model: 

- Creates a work-load environment by producing a series of entities, 
called jobs, with exponentially distributed inter-arrival time. 

Each job may queue up as backlog but eventually is assigned to 
system channels, where it passes through the various stages of log- 
ling in, loading, and repeated service cycles. 

- Creates a hardware configuration. The configuration is parameter- 
ized with respect to such factors as number of channels, core and 
drum size, and access and word transfer rates. Equipment configura- 
tion malfunctions are generated. The run duration, recording, and 
data reduction functions are also parameterized. 

- Generates events (job arrivals, computer malfunctions, and service 
requests at the channels) in simulated time with processing taking 
place on a next-event basis. 
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- Outputs both to the teletypewriter and to tape include a running 
account of major events to the teletype, permitting the user to 
monitor the run; more detailed recorded data directly to tape; job 
generation, completion, and back-log counts, yielding throughput 
information; individual job arrival and completion times to give 
some measure of turnaround time; demand and response-cycle distribu- 
tion to measure system operations; percentages of time spent on 
swapping, service, overhead, and idling, to measure computer use. 

NORAD Desk Top Exercises 

The NORAD Desk Top Exercises are designed to present a realistic, 
continent-wide battle simulation to the North American Air Defense 
System’s Combat Operations Center. They are also designed to actively 
exercise the entire NORAD system. Plans for the exercises are develop- 
ed jointly by NORAD and SDC. Designers and planners at SDC, working 
closely with NORAD headquarters, formulate and produce a comprehensive 
as well as detailed description of the Desk Top problem, a set of 
simulation procedures and the requisite problem input tapes, films 
and other aids for the entire exercise. A unique simulation facility, 
which includes seven large-scale computer-based simulation models, is 
used in the production of the inputs. During the exercises, simula- 
tion is supervised by local military personnel. The course of the 
exercise depends on the real-time reactions and inputs of NORAD 
personnel. SDC representatives are available at each facility for 
aid and consultation and also contribute extensively to the debrief- 
ings which follow the exercise. The Desk Top Exercises have evolved 
with the NORAD system from the comparatively small, manual Desk Top 
I to the huge, largely computerized Desk Top IX. 

Vehicular Traffic Model 



The Vehicular Traffic Study Project at SDC has developed a digital 
computer simulation model of a portion of a freeway diamond inter- 
change. The broad aim of the vehicular traffic study is to develop 
some much-needed basic knowledge that can be used to provide a tool 
for research on freeway diamond interchanges— research promoting 
improvement of the geometric design of the interchanges themselves 
and development of effective control devices regulating traffic flow 
through them. The method of the study is to simulate traffic flow in 
a diamond interchange between a freeway and an arterial street, and 
to validate this model by field data. 

Becuase of the sheer size of the physical area that must be examined, 
the number of vehicles involved, and the complexity of the task itself, 
the study has been divided into four phases or modules. 

- Merging of an on-ramp with freeway traffic; 

- Merging of an on-ramp with freeway traffic and an off-ramp with 
arterial traffic; 

- Processing of all arterial traffic; 
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- Simulation of the operation of the entire interchange by appropriate 
combination of modules 2 and 3 above. 

Strategic Mobility Planning Model (MORG-1 ) 

SDC assisted the National Military Command System Support Center in 
developing, designing, programming and implementing a strategic mobility 
planning model. MORG-1 provides rapid calculations of time-phased 
movement requirements for the delivery of forces and supplies into 
specific areas, with estimates of the availability dates and units 
necessary to meet those requirements . 

On a request from the Secretary of Defense to assemble a plan for 
deploying a specified force, the user will go to MORG-1 for information 
on the kinds and amounts of supplies that must accompany the force, 
and on where the supplies must be at a given time. MORG-1 will also 
yield information on the rate of supply that must be maintained during 
the force's deployment. 

Urban Planning Model 

The Los Angeles Department of City Planning has initiated a program of 
mathematical model development. The models, once developed, will 
assist city planners in the understanding of urban processes, and will 
also enable them to make better selections from among alternative plans. 
SDC is assisting the Department in the development and implementation 
of a residential location model. Simultaneous with model design, the 
needed data are being gathered from a variety of sources and converted 
to a form required by the model. 

National Policy Formulation 

SDC has developed a simulation vehicle for research purposes consist- 
ing of a socio-economic model of the American society, simulated major 
interest groups within that society, and exercise procedures enabling 
these groups to make decisions on given public policy issues during 
specified future time periods. Some examples of the policy issues 
examined in this research project have been the raising of tax 
exemptions; the reduction of the work week; the increase of federal aid 
to education; and the negotiation of arms control agreements. 

Simulation of Large Social Systems 

This research project (known as Leviathan) involved a large-scale 
simulation that represents the significant activities, organization, 
and communication patterns of hundreds of people. Functional responsi- 
bilities were specified for important decision-makers, whose roles 
were enacted by live subjects in one of SDC's laboratories. The 
subjects intervened in the computerized simulation through appropriate 
input /output and display equipment. 
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Data Processing System Simulator (DPSS) 



The DPSS was initially designed to meet the needs of analyzing and 
developing the data processing requirements of the Project 465L 
Strategic Air Command Control System. It has subsequently been general- 
ized to permit its application to other systems in various stages of 
design and development. Results have shown the usefulness of the 
DPSS in Project 465L and, in a preliminary manner, its usefulness on 
the Space Surveillance Project and the New York State Identification 
and Intelligence System. 

The Data Processing System Simulator is a general-purpose computer 
program that can be used for the evaluation cf a proposed new des n 
or a modification to an existing design of a data processing system 
prior to making equipment selections or performing any significant 
computer program design. The DPSS can also be used to provide guidance 
in the design and development of a data processing system during the 
detailed design stages. 

The DPSS can be used to determine the sensitivity of a data processing 
system's performance to various system loading or design parameters. 

In addition, the total system design, including the software and equip- 
ment portions, can be subjected to a rigorous analysis and evaluation 
early in the design process. 

The flexibility of the DPSS in terms of the variability of both the 
inputs to be tested and simulation program logic makes this tool use 
ful in the early stages of establishing data processing system require- 
ments. It is a powerful tool in performing system feasibility studies, 
simulating the operations and performance of a computer-based data 
processing system, and in evaluating equipment and data processing 
software combinations as a function of system operational require- 
ments. 
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